Image processing method and apparatus

ABSTRACT

This application provides an image processing method, including: adaptively determining at least one interpolation point based on feature information or dynamic metadata of a to-be-processed image, and determining the at least one interpolation point and a modified spline curve that passes through the at least one interpolation point, where the modified spline curve includes at least one of a linear spline curve or a cubic spline curve; and performing tone mapping on the to-be-processed image based on a modified tone mapping curve, where the modified tone mapping curve includes at least a part of an initial tone mapping curve and the modified spline curve. According to the method provided in this application, the tone mapping curve can be adaptively modified based on a scene requirement, so that images in various scenes can have a good display effect.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/CN2020/113525, filed on Sep. 4, 2020, which claims priority toChinese Patent Application No. 201910996393.9, filed on Oct. 18, 2019.The disclosures of the aforementioned applications are herebyincorporated by reference in their entireties.

TECHNICAL FIELD

This application relates to the field of image processing, and morespecifically, to an image processing method and apparatus.

BACKGROUND

A dynamic range (DR) may be used to represent a ratio of a maximumgrayscale value to a minimum grayscale value of a picture. The dynamicrange may be further used to represent a ratio of a maximum luminancevalue to a minimum luminance value of the picture. A dynamic range inthe real world is between 10⁻³ and 10⁶, and is referred to as a highdynamic range (HDR). A display capability of a display device isgenerally limited, and is far below a luminance range in the real world.For example, an object in the real world is bright, for example, aluminance value is 2000 nits. Maximum luminance at which the displaydevice can perform display is only 500 nits. If no processing isperformed, the display device cannot reproduce the object in the realworld. In another case, when an object in the real world is dark, forexample, a luminance value is 10 nits. Minimum luminance at which thedisplay device can perform display is 100 nits. If no processing isperformed, the display device cannot reproduce the object in the realworld. This application provides a method, so that the display devicevividly and truly reproduces a scene in the real world.

SUMMARY

This application provides an image processing method and apparatus, toimprove a tone mapping effect of a display device.

According to a first aspect, an image processing method is provided,including: determining a first coordinate of at least one interpolationpoint based on feature information or dynamic metadata of ato-be-processed image; determining at least one second coordinate and atleast one first-order derivative based on the first coordinate of the atleast one interpolation point; determining coefficients of a modifiedspline curve based on the first coordinate of the at least oneinterpolation point, the at least one second coordinate, and the atleast one first-order derivative, where the modified spline curveincludes at least one of a linear spline curve or a cubic spline curve;and performing tone mapping on the to-be-processed image based on amodified tone mapping curve, where the modified tone mapping curveincludes at least a part of an initial tone mapping curve and themodified spline curve.

A location of the interpolation point is adaptively determined based onthe feature information of the to-be-processed image or the dynamicmetadata of the to-be-processed image, for example, luminanceinformation and pixel distribution information, and a modified splinecurve that is most applicable to the to-be-processed image is flexiblydetermined based on the interpolation point. Tone mapping is performedon the to-be-processed image based on the modified tone mapping curve.Because the tone mapping curve is not unchanged but is adaptivelyadjusted based on the to-be-processed image or the dynamic metadata,images in various scenes may have a good display effect.

In the image processing method provided in this embodiment of thisapplication, a curve can be adaptively modified based on a scenerequirement, to achieve a specified effect, for example, increaseluminance, increase contrast, and blur partial information. For example,a display effect of a dark-region detail is improved by increasingdark-region luminance, and contrast may be increased by decreasingdark-region luminance, or a large quantity of bits may be allocated to abright region to display a bright-region detail.

In a possible implementation, the at least one interpolation pointincludes a first interpolation point, the modified spline curve includesa first spline curve, the first spline curve is a linear spline curvethat passes through the first interpolation point, and the coefficientsof the modified spline curve include a first coefficient MA0 and asecond coefficient MB0. The determining at least one second coordinateand at least one first-order derivative based on the first coordinate ofthe at least one interpolation point includes: determining a secondcoordinate VA1 of the first interpolation point on the initial tonemapping curve and a first-order derivative GD1 of the initial tonemapping curve at the first interpolation point based on a firstcoordinate TH1 of the first interpolation point. The determiningcoefficients of a modified spline curve based on the first coordinate ofthe at least one interpolation point, the at least one secondcoordinate, and the at least one first-order derivative includes:determining GD1 as a second coefficient MB0 of the first spline curve,and determining VA1 as a first coefficient MA0 of the first splinecurve.

In this embodiment of this application, only one interpolation pointneeds to be determined. Optionally, the first interpolation point isclose to a bright-region endpoint. This embodiment of this applicationcorresponds to a case in which an overall scene of an image is bright.In this case, the bright region is modified, to better display abright-region detail of the image. Specifically, tone mapping isperformed, based on a linear function, on a pixel region whose luminanceis greater than a preset threshold (TH1), and the first spline curve issmoothly connected to the initial tone mapping curve at the firstinterpolation point, to prevent an unsmooth tone mapping curve fromaffecting a tone mapping effect.

In a possible implementation, the at least one interpolation pointincludes a first interpolation point and a second interpolation point,the modified spline curve includes a first spline curve, the firstspline curve is a cubic spline curve that passes through the firstinterpolation point and the second interpolation point, and thecoefficients of the modified spline curve include a first coefficientMA1, a second coefficient MB1, a third coefficient MC1, and a fourthcoefficient MD1. The determining at least one second coordinate and atleast one first-order derivative based on the first coordinate of the atleast one interpolation point includes: respectively calculating secondcoordinates VA1 and VA2 of the first interpolation point and the secondinterpolation point on the initial tone mapping curve based on a firstcoordinate TH1 of the first interpolation point and a first coordinateTH2 of the second interpolation point; and calculating a first-orderderivative GD1 of the initial tone mapping curve at the firstinterpolation point and a first-order derivative GD2 of the initial tonemapping curve at the second interpolation point based on TH1 and TH2.The determining coefficients of a modified spline curve based on thefirst coordinate of the at least one interpolation point, the at leastone second coordinate, and the at least one first-order derivativeincludes: determining VA1 as a first coefficient MA1 of the first splinecurve; determining GD1 as a second coefficient MB1 of the first splinecurve; constructing, based on TH1, TH2, VA2, and GD2, a system ofequations that includes two equations; and solving the system ofequations to obtain a third coefficient MC1 and a fourth coefficient MD1of the first spline curve.

In this embodiment of this application, two interpolation points need tobe determined. In an optional case, the two interpolation points areclose to a bright-region endpoint. This embodiment of this applicationcorresponds to a case in which an overall scene of an image is bright.In this case, the bright region is modified, to better display abright-region detail of the image. Specifically, tone mapping isperformed on a pixel region between the first interpolation point andthe second interpolation point based on a cubic spline function, and thecubic spline function is smoothly connected to the initial tone mappingcurve at the first interpolation point, to prevent an unsmooth tonemapping curve from affecting a tone mapping effect.

In a possible implementation, the constructing, based on TH1, TH2, VA2,and GD2, a system of equations that includes two equations includes:enabling a second coordinate of the second interpolation point on thefirst spline curve to be a sum of VA2 and S1, to obtain a firstequation, where S1 is a coordinate offset value included in the dynamicmetadata; and enabling a first-order derivative of the first splinecurve at the second interpolation point to be a sum of GD2 and S2, toobtain a second equation, where S2 is a slope offset value included inthe dynamic metadata.

In a possible implementation, the at least one interpolation pointincludes a first interpolation point, the modified spline curve includesa first spline curve, the first spline curve is a linear spline functionthat passes through the origin and the first interpolation point, andthe coefficients of the modified spline curve include a first-orderderivative MB0 of the first spline curve. Before the determining atleast one second coordinate and at least one first-order derivativebased on the first coordinate of the at least one interpolation point,the method further includes:

obtaining the first-order derivative MB0 of the first spline curve basedon at least one of a maximum luminance value maxL of the to-be-processedimage and an average luminance value avgL of the to-be-processed imagethat are included in the dynamic metadata.

In a possible implementation, the at least one interpolation pointfurther includes a second interpolation point and a third interpolationpoint, the modified spline curve further includes a second spline curveand a third spline curve, the second spline curve is a cubic splinecurve that passes through the first interpolation point and the secondinterpolation point, the third spline curve is a cubic spline curve thatpasses through the second interpolation point and the thirdinterpolation point, and the coefficients of the modified spline curvefurther include a first coefficient MA1, a second coefficient MB1, athird coefficient MC1, and a fourth coefficient MD1 of the second splinecurve, and a first coefficient MA2, a second coefficient MB2, a thirdcoefficient MC2, and a fourth coefficient MD2 of the third spline curve.The determining at least one second coordinate and at least onefirst-order derivative based on the first coordinate of the at least oneinterpolation point includes: calculating a second coordinate VA1 of thefirst interpolation point on the first spline curve based on a firstcoordinate TH1 of the first interpolation point and MB0; calculating asecond coordinate VA3 of the third interpolation point on the initialtone mapping curve based on a first coordinate TH3 of the thirdinterpolation point; calculating a second coordinate VA2 of the secondinterpolation point on a first straight line based on a first coordinateTH2 of the second interpolation point, where the first straight line isa straight line on which the first interpolation point and the thirdinterpolation point are located; and calculating a first-orderderivative GD3 of the initial tone mapping curve at the thirdinterpolation point based on TH3.

In a possible implementation, the determining coefficients of a modifiedspline curve based on the first coordinate of the at least oneinterpolation point, the at least one second coordinate, and the atleast one first-order derivative includes: determining VA1 as the firstcoefficient MA1 of the second spline curve; determining MB0 as thesecond coefficient MB1 of the second spline curve; determining VA2 asthe first coefficient MA2 of the third spline curve;

constructing, based on TH1, TH2, TH3, VA2, VA3, GD3, and a derivativerelationship between the second spline curve and the third spline curveat the second interpolation point, a system of equations that includesfive equations; and solving the system of equations that includes fiveequations, to obtain MC1, MD1, MB2, MC2, and MD2.

This embodiment of this application corresponds to a case in which anoverall scene of an image is dark. In this case, a dark region ismodified, to better display a dark-region detail of the image, so that apixel whose luminance is lower than a display capability of a displaydevice can have a good display effect on the display device.Specifically, tone mapping is performed, based on a linear splinefunction formula, on pixels that are lower than a first threshold (TH1),so that dark-region detail information of the part is better reserved.Then, a smooth transition between the linear spline function and theinitial tone mapping curve is implemented based on a cubic splinefunction, to prevent an unsmooth tone mapping curve from affecting atone mapping effect. It should be understood that in this embodiment ofthis application, there are two cubic spline curves. In an optionalcase, there may be only one cubic spline curve, and the first splinecurve is smoothly connected to the initial tone mapping curve by usingone cubic spline function. In this case, only two interpolation pointsneed to be determined.

Specifically, the first spline curve increases dark-region luminancecompared with a base curve. This is equivalent to allocating more bitsto the dark region. The two cubic spline functions implement a smoothtransition between the first spline curve and the initial tone mappingcurve, and enhancement or weakening processing may be performed on amodified region by controlling a point on and a slope of the curve.

In a possible implementation, the constructing, based on TH1, TH2, TH3,VA2, VA3, GD3, and a derivative relationship between the second splinecurve and the third spline curve at the second interpolation point, asystem of equations that includes five equations includes: enabling asecond coordinate of the second interpolation point on the second splinecurve to be VA2, to obtain a first equation; enabling a secondcoordinate of the third interpolation point on the third spline curve tobe VA3, to obtain a second equation; enabling a first-order derivativeof the third spline curve at the third interpolation point to be GD3, toobtain a third equation; enabling first-order derivatives of the secondspline curve and the third spline curve at the second interpolationpoint to be equal, to obtain a fourth equation; and enablingsecond-order derivatives of the second spline curve and the third splinecurve at the second interpolation point to be equal, to obtain a fifthequation.

In a possible implementation, the determining a first coordinate of atleast one interpolation point based on feature information or dynamicmetadata of a to-be-processed image includes: determining the firstcoordinate TH1 of the first interpolation point based on the at leastone of the maximum luminance value maxL of the to-be-processed image andthe average luminance value avgL of the to-be-processed image that areincluded in the dynamic metadata; TH2=TH1+B; and TH3=TH2+C*TH2−D*CH2,where B, C, and D are constants. Optionally, B, C, and D are constants.In an optional case, the constant may be included in the dynamicmetadata. In an optional case, B, C, and D may be directly defined inrelated program code, do not need to be obtained from the dynamicmetadata.

In a possible implementation, the at least one interpolation pointincludes a first interpolation point, a second interpolation point, anda third interpolation point, the modified spline curve includes a firstspline curve and a second spline curve, the first spline curve is acubic spline curve that passes through the first interpolation point andthe second interpolation point, the second spline curve is a cubicspline curve that passes through the second interpolation point and thethird interpolation point, and the coefficients of the modified splinecurve include a first coefficient MA1, a second coefficient MB1, a thirdcoefficient MC1, and a fourth coefficient MD1 of the first spline curve,and a first coefficient MA2, a second coefficient MB2, a thirdcoefficient MC2, and a fourth coefficient MD2 of the second splinecurve. The determining at least one second coordinate and at least onefirst-order derivative based on the first coordinate of the at least oneinterpolation point includes: respectively calculating secondcoordinates VA1 and VA3 of the first interpolation point and the thirdinterpolation point on the initial tone mapping curve based on a firstcoordinate TH1 of the first interpolation point and a first coordinateTH3 of the third interpolation point; calculating a second coordinateVA2 of the second interpolation point on a first straight line based ona first coordinate TH2 of the second interpolation point, where thefirst straight line is a straight line on which the first interpolationpoint and the third interpolation point are located; and calculating afirst-order derivative GD1 of the initial tone mapping curve at thefirst interpolation point and a first-order derivative GD3 of theinitial tone mapping curve at the third interpolation point.

This embodiment of this application corresponds to a case in which animage includes a pixel located in a specific luminance range. In thiscase, the specific luminance range is mainly modified, to better displaya luminance detail of the image in the range. Specifically, tone mappingis performed on a pixel between the first interpolation point and thesecond interpolation point and a pixel between the second interpolationpoint and the third interpolation point based on a cubic splinefunction, to improve a display effect of the part of pixels. Further, asmooth transition exists between the first spline curve and the secondspline curve and between the initial tone mapping curve and each of thefirst spline curve and the second spline curve, to prevent an unsmoothtone mapping curve from affecting a tone mapping effect.

In a possible implementation, the determining coefficients of a modifiedspline curve based on the first coordinate of the at least oneinterpolation point, the at least one second coordinate, and the atleast one first-order derivative includes: determining VA1 as the firstcoefficient MA1 of the first spline curve; determining GD1 as the secondcoefficient MB1 of the first spline curve; determining a sum of VA2 andan offset value as the first coefficient MA2 of the second spline curve,where the offset value is 0, a positive number, or a negative number;constructing, based on TH1, TH2, TH3, VA2, VA3, GD3, and a derivativerelationship between the first spline curve and the second spline curveat the second interpolation point, a system of equations that includesfive equations; and solving the system of equations that includes fiveequations, to obtain MC1, MD1, MB2, MC2, and MD2.

Optionally, the offset value is included in the dynamic metadata.

In a possible implementation, the constructing, based on TH1, TH2, TH3,VA2, VA3, GD3, and a derivative relationship between the first splinecurve and the second spline curve at the second interpolation point, asystem of equations that includes five equations includes:

enabling a second coordinate of the second interpolation point on thefirst spline curve to be VA2, to obtain a first equation; enabling asecond coordinate of the third interpolation point on the second splinecurve to be VA3, to obtain a second equation; enabling a first-orderderivative of the second spline curve at the third interpolation pointto be GD3, to obtain a third equation; enabling first-order derivativesof the first spline curve and the second spline curve at the secondinterpolation point to be equal, to obtain a fourth equation; andenabling second-order derivatives of the first spline curve and thesecond spline curve at the second interpolation point to be equal, toobtain a fifth equation.

In a possible implementation, the at least one interpolation pointincludes a first interpolation point, a second interpolation point, anda third interpolation point, the modified spline curve includes a firstspline curve and a second spline curve, the first spline curve is acubic spline curve that passes through the first interpolation point andthe second interpolation point, the second spline curve is a linearspline curve that passes through the second interpolation point and thethird interpolation point, and the coefficients of the modified splinecurve include a first coefficient MA1, a second coefficient MB1, a thirdcoefficient MC1, and a fourth coefficient MD1 of the first spline curve,and a first coefficient MA2 and a second coefficient MB2 of the secondspline curve. The determining at least one second coordinate and atleast one first-order derivative based on the first coordinate of the atleast one interpolation point includes: respectively calculating secondcoordinates VA1 and VA3 of the first interpolation point and the thirdinterpolation point on the initial tone mapping curve based on a firstcoordinate TH1 of the first interpolation point and a first coordinateTH3 of the third interpolation point; calculating a second coordinateVA2 of the second interpolation point on a first straight line based ona first coordinate TH2 of the second interpolation point, where thefirst straight line is a straight line on which the first interpolationpoint and the third interpolation point are located; and calculating afirst-order derivative GD1 of the initial tone mapping curve at thefirst interpolation point.

In a possible implementation, the determining coefficients of a modifiedspline curve based on the first coordinate of the at least oneinterpolation point, the at least one second coordinate, and the atleast one first-order derivative includes: determining VA1 as the firstcoefficient MA1 of the first spline curve; determining GD1 as the secondcoefficient MB1 of the first spline curve; determining a sum of VA2 andan offset value as the first coefficient MA2 of the second spline curve,where the offset value is 0, a positive number, or a negative number;constructing, based on TH1, TH2, TH3, VA2, VA3, and a derivativerelationship between the first spline curve and the second spline curveat the second interpolation point, a system of equations that includesthree equations; and solving the system of equations that includes threeequations, to obtain MC1, MD1, and MB2.

It should be understood that the offset value may be included in thedynamic metadata.

This embodiment of this application corresponds to a case in which anoverall scene of an image is bright. In this case, a bright region ismodified, to better display a bright-region detail of the image.Specifically, tone mapping is performed on a pixel region between thesecond interpolation point and the third interpolation point based on alinear spline curve, and a linear spline function is smoothly connectedto the initial tone mapping curve based on a cubic spline functionbetween the first interpolation point and the second interpolationpoint, to prevent an unsmooth tone mapping curve from affecting a tonemapping effect.

In a possible implementation, the constructing, based on TH1, TH2, TH3,VA2, VA3, and a derivative relationship between the first spline curveand the second spline curve at the second interpolation point, a systemof equations that includes three equations includes: enabling a secondcoordinate of the second interpolation point on the first spline curveto be VA2, to obtain a first equation; enabling a second coordinate ofthe third interpolation point on the second spline curve to be VA3, toobtain a second equation; and enabling first-order derivatives of thefirst spline curve and the second spline curve at the secondinterpolation point to be equal, to obtain a third equation.

In a possible implementation, the at least one interpolation pointconstitutes at least one modified range, one modified range correspondsto one modified spline curve, and the performing tone mapping on theto-be-processed image based on a modified tone mapping curve includes:when a to-be-processed pixel is located in the at least one modifiedrange, performing tone mapping on the to-be-processed pixel based on themodified spline curve; or when the to-be-processed pixel is outside theat least one modified range, performing tone mapping on theto-be-processed pixel based on the initial tone mapping curve.

In a possible implementation, before the determining a first coordinateof at least one interpolation point based on feature information ordynamic metadata of a to-be-processed image, the method furtherincludes: determining a target scene mode applicable to theto-be-processed image based on a mode indication value in the dynamicmetadata, where the at least one interpolation point and the modifiedspline curve are related to the target scene mode.

For example, a quantity of determined interpolation points and aquantity of linear spline curves and cubic spline curves that areincluded in the modified spline curve are related to a scene modeapplicable to the to-be-processed image. When the overallto-be-processed image is dark, the first interpolation point is locatedin a region near a dark-region endpoint, and mapping is performed on apixel between the origin and the first interpolation point by using thelinear spline curve, so that the dark-region detail can be betterdisplayed. When the overall to-be-processed image is bright, a pluralityof interpolation points are located in a region near the bright-regionendpoint.

In a possible implementation, the initial tone mapping curve includesone of the following formulas:

${{L^{\prime} = {{a \times \left( \frac{p \times L}{{\left( {p - 1} \right) \times L} + 1} \right)^{m}} + b}};}{{{F(L)} = {{a \times \left( \frac{p \times L^{n}}{{\left( {{k_{1} \times p} - k_{2}} \right) \times L^{n}} + k_{3}} \right)^{m}} + b}};}{{L^{\prime} = {{a \times L} + b}};}{{L^{\prime} = \frac{c_{1} + {c_{2} \times L^{n}}}{1 + {c_{3} \times L^{n}}}};}{{L^{\prime} = {\sum_{\rho = 0}^{N}{\begin{pmatrix}N \\\rho\end{pmatrix} \times L^{\rho} \times \left( {1 - L} \right)^{N - \rho} \times P_{\rho}}}},}$

where L represents a linear signal value, L′ represents a non-linearsignal value, p, a, m, b, c₁, c₂, c₃, n, N, and ρ are all dynamic rangemapping coefficients, and 0≤L≤1.

In this embodiment of this application, the method provided in the firstaspect may be applied to a plurality of initial tone mapping curves, andis highly flexible.

In a possible implementation, the plurality of interpolation pointsinclude a first interpolation point, a second interpolation point, and athird interpolation point. The spline function includes a first splinefunction that passes through the first interpolation point and thesecond interpolation point, and a second spline function that passesthrough the second interpolation point and the third interpolationpoint. The first spline function includes a first coefficient MA1, asecond coefficient MB1, a third coefficient MC1, and a fourthcoefficient MD1. The second spline function includes a first coefficientMA2, a second coefficient MB2, a third coefficient MC2, and a fourthcoefficient MD2.

In a possible implementation, the first interpolation point is notlocated on the initial tone mapping curve, and determining a splinefunction that passes through the plurality of interpolation pointsincludes: determining a second coordinate VA1 of the first interpolationpoint on a linear spline function, where the linear spline function is alinear spline function between the origin and the first interpolationpoint; and determining VA1 as the first coefficient MA1 of the firstspline function.

In a possible implementation, the first interpolation point is locatedon the initial tone mapping curve, and determining a spline functionthat passes through the plurality of interpolation points includes:determining a second coordinate VA1 of the first interpolation point onthe initial tone mapping curve; and determining VA1 as the firstcoefficient MA1 of the first spline function.

In a possible implementation, determining a spline function that passesthrough the plurality of interpolation points further includes:determining a second coordinate VA3 of the third interpolation point onthe initial tone mapping curve; determining a second coordinate VA2 ofthe second interpolation point on a first straight line, where the firststraight line is a straight line on which the first interpolation pointand the third interpolation point are located; and determining VA2 asthe first coefficient MA2 of the second spline function.

In a possible implementation, the dynamic metadata further includes asecond coordinate offset of the second interpolation point, anddetermining a spline function that passes through the plurality ofinterpolation points further includes: determining a second coordinateVA3 of the third interpolation point on the initial tone mapping curve;determining a second coordinate VA2 of the second interpolation point ona first straight line, where the first straight line is a straight lineon which the first interpolation point and the third interpolation pointare located; and determining a sum of VA2 and the second coordinateoffset as the first coefficient MA2 of the second spline function.

In a possible implementation, determining a spline function that passesthrough the plurality of interpolation points further includes:determining a first-order derivative of the linear spline function asMB1; and determining a first-order derivative of the initial tonemapping curve at the third interpolation point as MB2.

In a possible implementation, determining a spline function that passesthrough the plurality of interpolation points further includes:determining a first-order derivative of the initial tone mapping curveat the first interpolation point as MB1; and determining a first-orderderivative of the initial tone mapping curve at the third interpolationpoint as MB2.

In a possible implementation, determining a spline function that passesthrough the plurality of interpolation points further includes: enablinga second coordinate value of the second interpolation point on the firstspline function to be VA2, to obtain a first equation; enabling a secondcoordinate value of the third interpolation point on the second splinefunction to be VA3, to obtain a second equation; enabling a first-orderderivative of the second spline function at the third interpolationpoint to be the first-order derivative of the initial tone mapping curveat the third interpolation point, to obtain a third equation; enablingfirst-order derivatives of the first spline function and the secondspline function at the second interpolation point to be equal, to obtaina fourth equation; enabling second-order derivatives of the first splinefunction and the second spline function at the second interpolationpoint to be equal, to obtain a fifth equation; and solving the firstequation, the second equation, the third equation, the fourth equation,and the fifth equation, to obtain the third coefficient MC1 and thefourth coefficient MD1 of the first spline function, and the thirdcoefficient MC2 and the fourth coefficient MD2 of the second splinefunction.

In a possible implementation, the first interpolation point and thethird interpolation point are located on the initial tone mapping curve,and determining a spline function that passes through the plurality ofinterpolation points includes: respectively determining secondcoordinate values VA1 and VA3 of the first interpolation point and thethird interpolation point on the initial tone mapping curve; determininga corresponding second coordinate value VA2 of the second interpolationpoint on a first straight line, where the first straight line is astraight line on which the first interpolation point and the thirdinterpolation point are located; determining VA1 as the firstcoefficient MA1 of the first spline function; determining VA2 as thefirst coefficient MA2 of the second spline function; respectivelydetermining slopes GD1 and GD3 of the initial tone mapping curve at thefirst interpolation point and the third interpolation point; enablingGD1 to be the second coefficient MB1 of the first spline function;enabling GD3 to be the second coefficient MB2 of the second splinefunction; enabling a second coordinate value of the second interpolationpoint on the first spline function to be VA2, to obtain a firstequation; enabling a second coordinate value of the third interpolationpoint on the second spline function to be VA3, to obtain a secondequation; enabling a first-order derivative of the second splinefunction at the third interpolation point to be GD3, to obtain a thirdequation; enabling first-order derivatives of the first spline functionand the second spline function at the second interpolation point to beequal, to obtain a fourth equation; enabling second-order derivatives ofthe first spline function and the second spline function at the secondinterpolation point to be equal, to obtain a fifth equation; solving thefirst equation, the second equation, the third equation, the fourthequation, and the fifth equation, to obtain the third coefficient MC1and the fourth coefficient MD1 of the first spline function, and thethird coefficient MC2 and the fourth coefficient MD2 of the secondspline function; determining the first spline function based on MA1,MB1, MC1, and MD1; and determining the second spline function based onMA2, MB2, MC2, and MD2.

In a possible implementation, the first interpolation point is notlocated on the initial tone mapping curve, and determining a splinefunction that passes through the plurality of interpolation pointsincludes: determining a second coordinate VA1 of the first interpolationpoint on a linear spline function as the first coefficient MA1 of thefirst spline function, where the linear spline function is a straightline that passes through the origin and the first interpolation point;

determining a second coordinate VA3 of the third interpolation point onthe initial tone mapping curve; determining a second coordinate VA2 ofthe second interpolation point on a first straight line as the firstcoefficient MA2 of the second spline function, where the first straightline is a straight line on which the first interpolation point and thethird interpolation point are located; determining a first-orderderivative of the linear spline function as MB1; determining afirst-order derivative of the initial tone mapping curve at the thirdinterpolation point as MB2; enabling a second coordinate value of thesecond interpolation point on the first spline function to be VA2, toobtain a first equation; enabling a second coordinate value of the thirdinterpolation point on the second spline function to be VA3, to obtain asecond equation; enabling a first-order derivative of the second splinefunction at the third interpolation point to be GD3, to obtain a thirdequation;

enabling first-order derivatives of the first spline function and thesecond spline function at the second interpolation point to be equal, toobtain a fourth equation; enabling second-order derivatives of the firstspline function and the second spline function at the secondinterpolation point to be equal, to obtain a fifth equation; solving thefirst equation, the second equation, the third equation, the fourthequation, and the fifth equation, to obtain the third coefficient MC1and the fourth coefficient MD1 of the first spline function, and thethird coefficient MC2 and the fourth coefficient MD2 of the secondspline function; determining the first spline function based on MA1,MB1, MC1, and MD1; and determining the second spline function based onMA2, MB2, MC2, and MD2.

In a possible implementation, the dynamic metadata includes a startlocation L1 of a range of the spline function and an end location L3 ofthe range of the spline function, and determining a plurality ofinterpolation points based on the dynamic metadata includes: TH1=L1/255,TH3=L3/255, and TH2=(TH1+TH3)/2. TH1 is a first coordinate of the firstinterpolation point, TH2 is a first coordinate of the secondinterpolation point, and TH3 is a first coordinate of the thirdinterpolation point.

In a possible implementation, determining a plurality of interpolationpoints based on luminance information of the target picture includes:determining the first coordinate TH1 of the first interpolation pointbased on the following formula:

${{TH}1} = \left\{ \begin{matrix}{p_{valueL}\ } & {{maxL}\  > {TPH}} \\{{p_{valueL} \times \left( \frac{\left( {{maxL} - {TPL}} \right)}{{TPH} - {TPL}} \right)} + {p_{valueH} \times \left( {1 - \left( \frac{\left( {{maxL} - {TPL}} \right)}{{TPH} - {TPL}} \right)} \right)\ }} & {{TPL} \leq \ {maxL}\  \leq {TPH}} \\{p_{valueH}\ } & {{maxL}\  < {TPL}}\end{matrix} \right.$

where maxL is included in the dynamic metadata, and maxL is a maximumluminance value of the to-be-processed image. When the to-be-processedimage is received, the first coordinate TH1 of the first interpolationpoint is calculated based on the foregoing formula and maxL included inthe dynamic metadata of the to-be-processed image. PvalueL is a firstthreshold of a dark-region pixel, PvalueH is a second threshold of thedark-region pixel, and the first threshold is less than the secondthreshold. For example, a value of PvalueL may be 0.15, and a value ofPvalueH may be 0.25. TPH is a third threshold of a bright-region pixel,and TPL is a fourth threshold of the bright-region pixel. For example, avalue of TPH may be 0.6, and a value of TPL may be 0.1. In an optionalcase, maxL in the foregoing formula may be replaced with avgL, or maxLmay be replaced with a*maxL+b*avgL, where a and b are constants, 0≤a≤1,and 0≤b≤1.

In a possible implementation, before the determining a second coordinateVA1 of the first interpolation point on a linear spline function, themethod further includes:

determining the first-order derivative MB0 of the linear splinefunction:

${{MB}0} = \left\{ \begin{matrix}{S_{valueL}\ } & {{aveL}\  > {AvH}} \\{{S_{valueL} \times \left( \frac{\left( {{avgL} - {AvL}} \right)}{{AvH} - {AvL}} \right)} + {S_{valueH} \times \left( {1 - \left( \frac{\left( {{avgL} - {AvL}} \right)}{{AvH} - {AvL}} \right)} \right)\ }} & {{AvL} \leq {avgL}\  \leq {AvH}} \\{S_{valueH}\ } & {{aveL}\  < {AvL}}\end{matrix} \right.$

where avgL is included in the dynamic metadata, avgL is an averageluminance value of the to-be-processed image. When the to-be-processedimage is received, a slope MB0 of the first spline curve is calculatedbased on the foregoing formula and avgL included in the dynamicmetadata. AvL is a fifth threshold of an average pixel, and AvH is asixth threshold of the average pixel. For example, a value of AvL may be0.15, and a value of AvH may be 0.65. SvalueL is a first slope thresholdfor luminance increase of a dark-region pixel, SvalueH is a second slopethreshold added to the luminance value of the dark-region pixel, and thefirst slope threshold is less than the second slope threshold. Forexample, a value of SvalueL may be 1, and a value of SvalueH may be 1.2.In an optional case, avgL in the foregoing formula may be replaced withmaxL, or may be replaced with a*maxL+b*avgL, where a and b areconstants, 0≤a≤1, and 0≤b≤1.

In a possible implementation, the first spline function f1 is asfollows:

f1=MD1×(L−TH1)³ +MC1×(L−TH1)² +MB1×(L−TH1)¹ +MA1

The second spline function f2 is as follows:

f2=MD2×(L−TH2)³ +MC2×(L−TH2)² +MB2×(L−TH2)¹ +MA2

TH1 is the first coordinate of the first interpolation point, TH2 is thefirst coordinate of the second interpolation point, and L is anindependent variable of a function.

In a possible implementation, performing, based on the spline function,tone mapping on a pixel that is in the target picture and that islocated in the range of the spline function includes: when theto-be-processed pixel is located between the first interpolation pointand the second interpolation point, performing tone mapping on theto-be-processed pixel based on the first spline function; and when theto-be-processed pixel is located between the second interpolation pointand the third interpolation point, performing tone mapping on theto-be-processed pixel based on the second spline function.

According to a second aspect, an image processing method is provided,including: determining a mode indication value in dynamic metadata,where the mode indication value is used to indicate a target scene modeapplicable to a to-be-processed image; invoking, based on the modeindication value, an obtaining process of a parameter that is of amodified spline curve and that corresponds to the target scene mode, andobtaining the modified spline curve; and performing tone mapping on theto-be-processed image based on a modified tone mapping curve, where themodified tone mapping curve includes at least a part of an initial tonemapping curve and the modified spline curve.

In the image processing method provided in this embodiment of thisapplication, a scene mode applicable to the to-be-processed image isdetermined based on the mode indication value in the dynamic metadata,and the initial tone mapping curve is properly and flexibly modified, sothat an image display effect can be adjusted for regions with differentluminance, to enhance a luminance change detail of the image, and bringbetter visual experience to a user. A plurality of styles of displayeffects such as softness, vividness, brightness, and dusk may beobtained by performing a plurality of forms of fine-tuning on the tonemapping curve.

In a possible implementation, the invoking, based on the mode indicationvalue, an obtaining process of a parameter that is of a modified splinecurve and that corresponds to the target scene mode includes: when themode indication value is a first preset value, invoking a firstobtaining process; when the mode indication value is a second presetvalue, invoking a second obtaining process; when the mode indicationvalue is a third preset value, invoking a third obtaining process; whenthe mode indication value is a fourth preset value, invoking a fourthobtaining process; or when the mode indication value is a fifth presetvalue, invoking a fifth obtaining process.

In a possible implementation, the modified spline curve obtained in thefirst obtaining process includes a first spline curve, a second splinecurve, and a third spline curve, the first spline curve is a linearspline function that passes through an origin and a first interpolationpoint, the second spline curve is a cubic spline curve that passesthrough the first interpolation point and a second interpolation point,the third spline curve is a cubic spline curve that passes through thesecond interpolation point and a third interpolation point, and theinvoking a first obtaining process includes: determining a firstcoordinate TH1 of the first interpolation point based on at least one ofa maximum luminance value maxL of the to-be-processed image and anaverage luminance value avgL of the to-be-processed image that areincluded in the dynamic metadata; obtaining a first-order derivative MB0of the first spline curve based on the at least one of the averageluminance value avgL of the to-be-processed image and the maximumluminance value maxL of the to-be-processed image that are included inthe dynamic metadata; TH2=TH1+B; TH3=TH2+C*TH2−D*CH2, where TH2 is afirst coordinate of the second interpolation point, TH3 is a firstcoordinate of the third interpolation point, B, C, and D are constants,and optionally, B, C, and D may be carried in the dynamic metadata;calculating a second coordinate VA1 of the first interpolation point onthe first spline curve based on TH1 and MB0; calculating a secondcoordinate VA3 of the third interpolation point on the initial tonemapping curve based on TH3; calculating a second coordinate VA2 of thesecond interpolation point on a first straight line based on TH2, wherethe first straight line is a straight line on which the firstinterpolation point and the third interpolation point are located;calculating a first-order derivative GD3 of the initial tone mappingcurve at the third interpolation point based on TH3; determining VA1 asa first coefficient MA1 of the second spline curve; determining MB0 as asecond coefficient MB1 of the second spline curve; determining a sum ofVA2 and an offset value as a first coefficient MA2 of the third splinecurve, where the offset value is 0, a positive number, or a negativenumber; constructing, based on TH1, TH2, TH3, VA2, VA3, GD3, and aderivative relationship between the second spline curve and the thirdspline curve at the second interpolation point, a system of equationsthat includes five equations; and solving the system of equations thatincludes five equations, to obtain MC1, MD1, MB2, MC2, and MD2.

The first obtaining process corresponds to a case in which an overallscene of an image is dark. In this case, a dark region is modified, tobetter display a dark-region detail of the image, so that a pixel whoseluminance is lower than a display capability of a display device canhave a good display effect on the display device. Specifically, tonemapping is performed, based on a linear spline function formula, onpixels that are lower than a first threshold (TH1), so that dark-regiondetail information of the part is better reserved. Then, a smoothtransition between the linear spline function and the initial tonemapping curve is implemented based on a cubic spline function, toprevent an unsmooth tone mapping curve from affecting a tone mappingeffect. It should be understood that in this embodiment of thisapplication, there are two cubic spline curves. In an optional case,there may be only one cubic spline curve, and the first spline curve issmoothly connected to the initial tone mapping curve by using one cubicspline function. In this case, only two interpolation points need to bedetermined.

The first spline curve increases dark-region luminance compared with abase curve. This is equivalent to allocating more bits to the darkregion. The two cubic spline functions implement a smooth transitionbetween the first spline curve and the initial tone mapping curve, andenhancement or weakening processing may be performed on a modifiedregion by controlling a point on and a slope of the curve.

In a possible implementation, the modified spline curve obtained in thesecond obtaining process includes a first spline curve and a secondspline curve, the first spline curve is a cubic spline curve that passesthrough a first interpolation point and a second interpolation point,the second spline curve is a cubic spline curve that passes through thesecond interpolation point and a third interpolation point, and theinvoking a second obtaining process includes: obtaining a firstcoordinate TH1 of the first interpolation point, a first coordinate TH2of the second interpolation point, and a first coordinate TH3 of thethird interpolation point based on the dynamic metadata; respectivelycalculating second coordinates VA1 and VA3 of the first interpolationpoint and the third interpolation point on the initial tone mappingcurve based on TH1 and TH3; calculating a second coordinate VA2 of thesecond interpolation point on a first straight line based on TH2, wherethe first straight line is a straight line on which the firstinterpolation point and the third interpolation point are located;calculating a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point and a first-order derivative GD3of the initial tone mapping curve at the third interpolation point;determining VA1 as a first coefficient MA1 of the first spline curve;determining GD1 as a second coefficient MB1 of the first spline curve;determining a sum of VA2 and an offset value as a first coefficient MA2of the second spline curve, where the offset value is 0, a positivenumber, or a negative number; constructing, based on TH1, TH2, TH3, VA2,VA3, GD3, and a derivative relationship between the first spline curveand the second spline curve at the second interpolation point, a systemof equations that includes five equations; and solving the system ofequations that includes five equations, to obtain MC1, MD1, MB2, MC2,and MD2.

The second obtaining process corresponds to a case in which an imageincludes a pixel located in a specific luminance range. In this case,the specific luminance range is mainly modified, to better display aluminance detail of the image in the range. Specifically, tone mappingis performed on a pixel between the first interpolation point and thesecond interpolation point and a pixel between the second interpolationpoint and the third interpolation point based on a cubic splinefunction, to improve a display effect of the part of pixels. Further, asmooth transition exists between the first spline curve and the secondspline curve and between the initial tone mapping curve and each of thefirst spline curve and the second spline curve, to prevent an unsmoothtone mapping curve from affecting a tone mapping effect.

Specifically, in this modification manner, any range may be modified. Inaddition, a modified curve of the modified range is smoothly connectedto the initial tone mapping curve, and an interpolation point and anoffset value may be controlled to increase image contrast or increase ordecrease luminance values of a part of pixels, to achieve a desiredeffect.

In a possible implementation, the modified spline curve obtained in thethird obtaining process includes a first spline curve, the first splinecurve is a linear spline curve that passes through a first interpolationpoint, and the invoking a third obtaining process includes: obtaining afirst coordinate TH1 of the first interpolation point based on thedynamic metadata; determining a second coordinate VA1 of the firstinterpolation point on the initial tone mapping curve and a first-orderderivative GD1 of the initial tone mapping curve at the firstinterpolation point based on TH1; determining GD1 as a secondcoefficient MB0 of the first spline curve; and determining VA1 as afirst coefficient MAO of the first spline curve.

The third obtaining process corresponds to a case in which an overallscene of an image is bright. In this case, a bright region is modified,to better display a bright-region detail of the image. Specifically,tone mapping is performed, based on a linear function, on a pixel regionthat is greater than a preset threshold (TH1), and the first splinecurve is smoothly connected to the initial tone mapping curve at thefirst interpolation point, to prevent an unsmooth tone mapping curvefrom affecting a tone mapping effect.

Specifically, when tone mapping is performed, by using a linear splinecurve, on a pixel whose value is greater than the first interpolation ofthe first interpolation point, it indicates that all pixels whose valueis greater than TH1 are compressed by using a compression rate of thefirst interpolation point, to avoid excessive compression of abright-region pixel, and improve an image luminance value.

In a possible implementation, the modified spline curve obtained in thefourth obtaining process includes a first spline curve, the first splinecurve is a cubic spline curve that passes through a first interpolationpoint and a second interpolation point, and the invoking a fourthobtaining process includes: obtaining a first coordinate TH1 of thefirst interpolation point and a first coordinate TH2 of the secondinterpolation point based on the dynamic metadata; respectivelycalculating second coordinates VA1 and VA2 of the first interpolationpoint and the second interpolation point on the initial tone mappingcurve based on TH1 and TH2; calculating a first-order derivative GD1 ofthe initial tone mapping curve at the first interpolation point and afirst-order derivative GD2 of the initial tone mapping curve at thesecond interpolation point based on TH1 and TH2; enabling a secondcoordinate of the second interpolation point on the first spline curveto be a sum of VA2 and S1, to obtain a first equation, where S1 is acoordinate offset value included in the dynamic metadata; enabling afirst-order derivative of the first spline curve at the secondinterpolation point to be a sum of GD2 and S2, to obtain a secondequation, where S2 is a slope offset value included in the dynamicmetadata; and solving a system of equations that includes the firstequation and the second equation, to obtain a third coefficient MC1 anda fourth coefficient MD1 of the first spline curve.

The fourth obtaining process corresponds to a case in which an overallscene of an image is bright. In this case, a bright region is modified,to better display a bright-region detail of the image. Specifically,tone mapping is performed on a pixel region between the firstinterpolation point and the second interpolation point based on a cubicspline function, and the cubic spline function is smoothly connected tothe initial tone mapping curve at the first interpolation point, toprevent an unsmooth tone mapping curve from affecting a tone mappingeffect.

Specifically, a luminance value of a pixel between the firstinterpolation point and a bright-region pixel endpoint may be increased,or a luminance value of a pixel between the first interpolation pointand a bright-region pixel endpoint may be decreased. This may achieve amore dazzling effect by increasing luminance, and may enhance thebright-region detail by decreasing luminance. In addition, the increaseand the decrease are nonlinear processes, so that contrast can beincreased.

In a possible implementation, the modified spline curve obtained in thefifth obtaining process includes a first spline curve and a secondspline curve, the first spline curve is a cubic spline curve that passesthrough a first interpolation point and a second interpolation point,the second spline curve is a linear spline curve that passes through thesecond interpolation point and a third interpolation point, and theinvoking a fifth obtaining process includes: obtaining a firstcoordinate TH1 of the first interpolation point, a first coordinate TH2of the second interpolation point, and a first coordinate TH3 of thethird interpolation point based on the dynamic metadata; respectivelycalculating second coordinates VA1 and VA3 of the first interpolationpoint and the third interpolation point on the initial tone mappingcurve based on TH1 and TH3; calculating a second coordinate VA2 of thesecond interpolation point on a first straight line based on TH2, wherethe first straight line is a straight line on which the firstinterpolation point and the third interpolation point are located;calculating a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point; determining VA1 as a firstcoefficient MA1 of the first spline curve; determining GD1 as a secondcoefficient MB1 of the first spline curve; determining, as a firstcoefficient MA2 of the second spline curve, a sum of VA2 and acoordinate offset value included in the dynamic metadata; constructing,based on TH1, TH2, TH3, VA2, VA3, and a derivative relationship betweenthe first spline curve and the second spline curve at the secondinterpolation point, a system of equations that includes threeequations; and solving the system of equations that includes threeequations, to obtain MC1, MD1, and MB2.

In an optional case, a luminance value of the first interpolation pointis greater than a luminance value of the second interpolation point, andthe luminance value of the second interpolation point is greater than aluminance value of the third interpolation point.

In an optional case, a luminance value of the first interpolation pointis less than a luminance value of the second interpolation point, andthe luminance value of the second interpolation point is less than aluminance value of the third interpolation point.

The fifth obtaining process corresponds to a case in which an overallscene of an image is bright. In this case, a bright region is modified,to better display a bright-region detail of the image. Specifically,tone mapping is performed on a pixel region between the secondinterpolation point and the third interpolation point based on a linearspline curve, and a linear spline function is smoothly connected to theinitial tone mapping curve based on a cubic spline function between thefirst interpolation point and the second interpolation point, to preventan unsmooth tone mapping curve from affecting a tone mapping effect.

Specifically, in this scenario, a cubic spline function may be usedbetween the first interpolation point and the second interpolation pointto increase luminance and decrease luminance for the initial tonemapping curve, and the increase and the decrease are nonlinear. Tonemapping is performed between the second interpolation point and thethird interpolation point based on a straight line, so that more bitscan be allocated to a pixel between the second interpolation point andthe third interpolation point, to enhance the bright-region detail.

In a possible implementation, the initial tone mapping curve includesone of the following formulas:

${{L^{\prime} = {{a \times \left( \frac{p \times L}{{\left( {p - 1} \right) \times L} + 1} \right)^{m}} + b}};}{{L^{\prime} = {{a \times L} + b}};}{{L^{\prime} = \frac{c_{1} + {c_{2} \times L^{n}}}{1 + {c_{3} \times L^{n}}}};}{{L^{\prime} = {\sum_{\rho = 0}^{N}{\begin{pmatrix}N \\\rho\end{pmatrix} \times L^{\rho} \times \left( {1 - L} \right)^{N - \rho} \times P_{\rho}}}},}$

where L represents a linear signal value, L′ represents a non-linearsignal value, p, a, m, b, ci, c2, c3, n, N, and p are all dynamic rangemapping coefficients, and 0<L<1.

According to a third aspect, this application provides an imageprocessing apparatus, including a module configured to perform themethod according to any one of the first aspect or the possibleimplementations of the first aspect.

According to a fourth aspect, this application provides an imageprocessing apparatus, including: a determining module, configured todetermine a mode indication value in dynamic metadata, where the modeindication value is used to indicate a target scene mode applicable to ato-be-processed image, where the determining module is furtherconfigured to: invoke, based on the mode indication value, an obtainingprocess of a parameter that is of a modified spline curve and thatcorresponds to the target scene mode, and obtain the modified splinecurve; and a tone mapping module, configured to perform tone mapping onthe to-be-processed image based on a modified tone mapping curve, wherethe modified tone mapping curve includes at least a part of an initialtone mapping curve and the modified spline curve.

In a possible implementation, the determining module is specificallyconfigured to: when the mode indication value is a first preset value,invoke a first obtaining process; when the mode indication value is asecond preset value, invoke a second obtaining process; when the modeindication value is a third preset value, invoke a third obtainingprocess; when the mode indication value is a fourth preset value, invokea fourth obtaining process; or when the mode indication value is a fifthpreset value, invoke a fifth obtaining process.

In a possible implementation, the modified spline curve obtained in thefirst obtaining process includes a first spline curve, a second splinecurve, and a third spline curve, the first spline curve is a linearspline function that passes through an origin and a first interpolationpoint, the second spline curve is a cubic spline curve that passesthrough the first interpolation point and a second interpolation point,the third spline curve is a cubic spline curve that passes through thesecond interpolation point and a third interpolation point, and thedetermining module is specifically configured to: determine a firstcoordinate TH1 of the first interpolation point based on at least one ofa maximum luminance value maxL of the to-be-processed image and anaverage luminance value avgL of the to-be-processed image that areincluded in the dynamic metadata; obtain a first-order derivative MB0 ofthe first spline curve based on the at least one of the maximumluminance value maxL of the to-be-processed image and the averageluminance value avgL of the to-be-processed image that are included inthe dynamic metadata; TH2=TH1 +B; TH3=TH2 +C*TH2−D*CH2, where TH2 is afirst coordinate of the second interpolation point, TH3 is a firstcoordinate of the third interpolation point, B, C, and D are constants,and optionally, B, C, and D are constants included in the dynamicmetadata; calculate a second coordinate VA1 of the first interpolationpoint on the first spline curve based on TH1 and MB0; calculate a secondcoordinate VA3 of the third interpolation point on the initial tonemapping curve based on TH3; calculate a second coordinate VA2 of thesecond interpolation point on a first straight line based on TH2, wherethe first straight line is a straight line on which the firstinterpolation point and the third interpolation point are located;calculate a first-order derivative GD3 of the initial tone mapping curveat the third interpolation point based on TH3; determine VA1 as a firstcoefficient MA1 of the second spline curve; determine MB0 as a secondcoefficient MB1 of the second spline curve; determine a sum of VA2 andan offset value as a first coefficient

MA2 of the third spline curve, where the offset value is 0, a positivenumber, or a negative number; construct, based on TH1, TH2, TH3, VA2,VA3, GD3, and a derivative relationship between the second spline curveand the third spline curve at the second interpolation point, a systemof equations that includes five equations; and solve the system ofequations that includes five equations, to obtain MC1, MD1, MB2, MC2,and MD2.

In a possible implementation, the modified spline curve obtained in thesecond obtaining process includes a first spline curve and a secondspline curve, the first spline curve is a cubic spline curve that passesthrough a first interpolation point and a second interpolation point,the second spline curve is a cubic spline curve that passes through thesecond interpolation point and a third interpolation point, and thedetermining module is specifically configured to: obtain a firstcoordinate TH1 of the first interpolation point, a first coordinate TH2of the second interpolation point, and a first coordinate TH3 of thethird interpolation point based on the dynamic metadata; respectivelycalculate second coordinates VA1 and VA3 of the first interpolationpoint and the third interpolation point on the initial tone mappingcurve based on TH1 and TH3; calculate a second coordinate VA2 of thesecond interpolation point on a first straight line based on TH2, wherethe first straight line is a straight line on which the firstinterpolation point and the third interpolation point are located;calculate a first-order derivative GD1 of the initial tone mapping curveat the first interpolation point and a first-order derivative GD3 of theinitial tone mapping curve at the third interpolation point; determineVA1 as a first coefficient MA1 of the first spline curve; determine GD1as a second coefficient MB1 of the first spline curve; determine a sumof VA2 and an offset value as a first coefficient MA2 of the secondspline curve, where the offset value is 0, a positive number, or anegative number; construct, based on TH1, TH2, TH3, VA2, VA3, GD3, and aderivative relationship between the first spline curve and the secondspline curve at the second interpolation point, a system of equationsthat includes five equations; and solve the system of equations thatincludes five equations, to obtain MC1, MD1, MB2, MC2, and MD2.

In a possible implementation, the modified spline curve obtained in thethird obtaining process includes a first spline curve, the first splinecurve is a linear spline curve that passes through a first interpolationpoint, and the determining module is specifically configured to: obtaina first coordinate TH1 of the first interpolation point based on thedynamic metadata; determine a second coordinate VA1 of the firstinterpolation point on the initial tone mapping curve and a first-orderderivative GD1 of the initial tone mapping curve at the firstinterpolation point based on TH1; determine GD1 as a second coefficientMB0 of the first spline curve; and determine VA1 as a first coefficientMAO of the first spline curve.

In a possible implementation, the modified spline curve obtained in thefourth obtaining process includes a first spline curve, the first splinecurve is a cubic spline curve that passes through a first interpolationpoint and a second interpolation point, and the determining module isspecifically configured to: obtain a first coordinate TH1 of the firstinterpolation point and a first coordinate TH2 of the secondinterpolation point based on the dynamic metadata; respectivelycalculate second coordinates VA1 and VA2 of the first interpolationpoint and the second interpolation point on the initial tone mappingcurve based on TH1 and TH2; calculate a first-order derivative GD1 ofthe initial tone mapping curve at the first interpolation point and afirst-order derivative GD2 of the initial tone mapping curve at thesecond interpolation point based on TH1 and TH2; enable a secondcoordinate of the second interpolation point on the first spline curveto be a sum of VA2 and S1, to obtain a first equation, where S1 is acoordinate offset value included in the dynamic metadata; enable afirst-order derivative of the first spline curve at the secondinterpolation point to be a sum of GD2 and S2, to obtain a secondequation, where S2 is a slope offset value included in the dynamicmetadata; and solve a system of equations that includes the firstequation and the second equation, to obtain a third coefficient MC1 anda fourth coefficient MD1 of the first spline curve.

In a possible implementation, the modified spline curve obtained in thefifth obtaining process includes a first spline curve and a secondspline curve, the first spline curve is a cubic spline curve that passesthrough a first interpolation point and a second interpolation point,the second spline curve is a linear spline curve that passes through thesecond interpolation point and a third interpolation point, and thedetermining module is specifically configured to: obtain a firstcoordinate TH1 of the first interpolation point, a first coordinate TH2of the second interpolation point, and a first coordinate TH3 of thethird interpolation point based on the dynamic metadata; respectivelycalculate second coordinates VA1 and VA3 of the first interpolationpoint and the third interpolation point on the initial tone mappingcurve based on TH1 and TH3; calculate a second coordinate VA2 of thesecond interpolation point on a first straight line based on TH2, wherethe first straight line is a straight line on which the firstinterpolation point and the third interpolation point are located;calculate a first-order derivative GD1 of the initial tone mapping curveat the first interpolation point; determine VA1 as a first coefficientMA1 of the first spline curve;

determine GD1 as a second coefficient MB1 of the first spline curve;determine a sum of VA2 and an offset value as a first coefficient MA2 ofthe second spline curve, where the offset value is 0, a positive number,or a negative number; construct, based on TH1, TH2, TH3, VA2, VA3, and aderivative relationship between the first spline curve and the secondspline curve at the second interpolation point, a system of equationsthat includes three equations; and solve the system of equations thatincludes three equations, to obtain MC1, MD1, and MB2.

According to a fifth aspect, this application provides an imageprocessing apparatus, including a processor and a transmissioninterface. The transmission interface is configured to receive or sendimage data. The processor is configured to: invoke program instructionsstored in a memory, and execute the program instructions to perform themethod according to any one of the first aspect or the implementationsof the first aspect.

For example, the image processing apparatus may be a chip.

According to a sixth aspect, this application provides an imageprocessing apparatus, including a processor and a transmissioninterface. The transmission interface is configured to receive or sendimage data. The processor is configured to: invoke program instructionsstored in a memory, and execute the program instructions to perform themethod according to any one of the first aspect or the possibleimplementations of the first aspect.

In a possible implementation, the image processing apparatus may furtherinclude a memory.

According to a seventh aspect, this application provides an imageprocessing apparatus, including a processor and a transmissioninterface. The transmission interface is configured to receive or sendimage data. The processor is configured to: invoke program instructionsstored in a memory, and execute the program instructions to perform themethod according to any one of the second aspect or the possibleimplementations of the second aspect.

According to a eighth aspect, this application provides an imageprocessing apparatus, including a processor and a memory, and configuredto perform the method according to any one of the first aspect or theimplementations of the first aspect.

According to an ninth aspect, this application provides an imageprocessing apparatus, including a processor and a memory, and configuredto perform the method according to any one of the second aspect or theimplementations of the second aspect.

According to a tenth aspect, this application provides acomputer-readable storage medium. The computer-readable storage mediumstores program instructions, and when the program instructions are runon a computer or a processor, the computer or the processor is enabledto perform the method according to any one of the first aspect or theimplementations of the first aspect.

According to a eleventh aspect, this application provides acomputer-readable storage medium. The computer-readable storage mediumstores program instructions, and when the program instructions are runon a computer or a processor, the computer or the processor is enabledto perform the method according to any one of the second aspect or theimplementations of the second aspect.

According to an twelfth aspect, this application provides a computerprogram product, including a computer program. When the computer programis executed on a computer or a processor, the computer or the processoris enabled to perform the method according to any one of the firstaspect or the implementations of the first aspect.

According to a thirteenth aspect, this application provides a computerprogram product, including a computer program. When the computer programis executed on a computer or a processor, the computer or the processoris enabled to perform the method according to any one of the secondaspect or the implementations of the second aspect.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic block diagram of an image display system accordingto an embodiment of this application;

FIG. 2 is a diagram of a hardware architecture of an example imageprocessing apparatus according to an embodiment of this application;

FIG. 3 is a schematic flowchart of an example image processing methodaccording to an embodiment of this application;

FIG. 4a is a schematic diagram of an example modified spline curveaccording to an embodiment of this application;

FIG. 4b is a schematic diagram of an example modified spline curveaccording to an embodiment of this application;

FIG. 4c is a schematic diagram of an example modified spline curveaccording to an embodiment of this application;

FIG. 4d is a schematic diagram of an example modified spline curveaccording to an embodiment of this application;

FIG. 4e is a schematic diagram of an example modified spline curveaccording to an embodiment of this application;

FIG. 4f is a schematic diagram of several example initial tone mappingcurves according to an embodiment of this application;

FIG. 5 is a flowchart of an example method for obtaining a parameter ofa modified spline curve according to an embodiment of this application;

FIG. 6a is a flowchart of an example method for obtaining a parameter ofa modified spline curve according to an embodiment of this application;

FIG. 6b is a flowchart of an example method for obtaining a parameter ofa modified spline curve according to an embodiment of this application;

FIG. 7 is a flowchart of an example method for obtaining a parameter ofa modified spline curve according to an embodiment of this application;

FIG. 8 is a flowchart of an example method for obtaining a parameter ofa modified spline curve according to an embodiment of this application;

FIG. 9 is a flowchart of an example method for obtaining a parameter ofa modified spline curve according to an embodiment of this application;

FIG. 10 is a flowchart of an example method for obtaining a parameter ofa modified spline curve according to an embodiment of this application;and

FIG. 11 is a structural block diagram of an example image processingapparatus according to an embodiment of this application.

DESCRIPTION OF EMBODIMENTS

In the specification, claims, and accompanying drawings of thisapplication, the terms “first”, “second”, and the like are intended todistinguish between similar objects but do not necessarily indicate aspecific order or sequence. Moreover, the terms “include”, “have”, andany other variant thereof are intended to cover a non-exclusiveinclusion, for example, including a series of steps or units. Methods,systems, products, or devices are not necessarily limited to thoseexpressly listed steps or units, but may include other steps or unitsthat are not expressly listed or that are inherent to such processes,methods, products, or devices.

It should be understood that, in this application, “at least one” meansone or more, and “a plurality of” means two or more. The term “and/or”is used to describe an association relationship between associatedobjects, and indicates that three relationships may exist. For example,“A and/or B” may indicate the following three cases: Only A exists, onlyB exists, and both A and B exist, where A and B may be singular orplural. The character “I” generally indicates an “or” relationshipbetween the associated objects. “At least one of the following” orsimilar expressions indicate any combination of the following, includingone or any combination of two or more of the following. For example, atleast one of a, b, or c may indicate: a, b, c, “a and b”, “a and c”, “band c”, or “a, b and c”, where there may one or more a-s, one or moreb-s, and one or more c-s.

An image obtaining device may generate, by receiving an optical signalin a natural scene, a picture or a video that includes the naturalscene. To facilitate image or video transmission, the optical signalneeds to be converted into an electrical signal, and image informationof each pixel is recorded by using a luminance value or a chrominancevalue (for example, a grayscale value is between 0 and 255) in a fixedvalue range. A curve for converting the optical signal into theelectrical signal is an optical-electro transfer function. Commonoptical-electro transfer functions include a PQ (perceptual quantizer)optical-electro transfer function, an HLG (hybrid log-gamma)optical-electro transfer function, and a scene luminance fidelity (SLF)optical-electro transfer function. A display device may reproduce thenatural scene by displaying the picture or the video. In an optionalcase, the image obtaining device is referred to as a front-end device,and the display device is referred to as a back-end device. The displaydevice may determine, based on the optical-electro transfer function anda luminance value or a grayscale value of each pixel in an image,luminance that is of an object and that exists during photographing,that is, an electronic device may convert YUV or RGB information intoluminance in a unit of nit. However, luminance of the object may exceeda display capability of the display device. Because luminanceinformation obtained by the image obtaining device does not match aluminance display capability of the display device, the following casesexist: A display device with a low luminance display capability displaysa high-luminance image, and a display device with a high luminancedisplay capability displays a low-luminance image. In these cases, tonemapping (tone-mapping) needs to be performed on the image obtained bythe image obtaining device, so that the image matches the displaycapability of the display device. For example, tone mapping may beperformed on a high dynamic range image based on a tone mapping curve,to display the image on a display device with a low dynamic rangedisplay capability. It should be understood that the tone mapping curvemay be determined by the display device, or may be determined by aprocessing device disposed outside the display device.

However, in some cases, when the tone mapping curve is not accurate, adisplay effect of the high dynamic range image is poor. For example, ina process of playing a video, if only one tone mapping curve isdetermined for the video, a difference between image content displayedin different video frames may be large. For example, images displayed insome video frames are very dark, and images displayed in some othervideo frames are very bright. If a fixed tone mapping curve is used forthe entire video, no frame of picture in the video can be perfectlyreproduced on the display device. In addition, the tone mapping curvecarries a large amount data. If a tone mapping curve is determined foreach image or picture frame, a data redundancy result may be caused.

This application provides an image processing method and apparatus, toflexibly modify an initial tone mapping curve based on a scene feature,so that a good display effect or a display effect that meets asubjective requirement can be obtained, in different scenes, for animage on which tone mapping is performed.

FIG. 1 is a schematic block diagram of an example image display system10 according to an embodiment of this application. The image displaysystem 10 may include an image obtaining device 12, an image processingdevice 14, and a display device 34. For example, the image obtainingdevice 12 may obtain an HDR image or a standard dynamic range (StandardDynamic Range, SDR) image. The image processing device 14 may processthe HDR image or the SDR image obtained by the image obtaining device12.

The image obtaining device 12 and the image processing device 14 may becommunicatively connected by using a link 13, and the image processingdevice 14 may receive image data from the image obtaining device 12 byusing the link 13. The link 13 may include one or more communicationmedia or apparatuses. The one or more communication media may include awireless communication medium and/or a wired communication medium, suchas a radio frequency (RF) spectrum or one or more physical transmissionlines. Optionally, the one or more communication media may form a partof a packet-based network, and the packet-based network may be, forexample, a local area network, a wide area network, or a global network(for example, the Internet). The one or more communication media mayinclude a router, a switch, a base station, or another device thatfacilitates communication from the image obtaining device 12 to theimage processing device 14. In an optional case, the link 13 may be aBluetooth wireless link.

For example, the image obtaining device 12 includes a picture source 16and a communication interface 22 that may be alternatively referred toas a transmission interface. In addition, optionally, the imageobtaining device 12 may further include an image processor 20. In aspecific implementation, the image processor 20, the picture source 16,and the communication interface 22 may be hardware components in theimage obtaining device 12, or may be software programs in the imageobtaining device 12, or the communication interface 22 may be acombination of a hardware module and a software program.

The picture source 16 may include or be any type of picture capturingdevice configured to, for example, capture a real-world picture; and/orany type of device for generating a picture or comment (for screencontent encoding, some text on a screen is also considered as a part ofa to-be-encoded picture or image), for example, a computer graphicsprocessor configured to generate a computer animation picture; or anytype of device configured to obtain and/or provide a real-world pictureor a computer animation picture (for example, screen content or avirtual reality (VR) picture); and/or any combination thereof (forexample, an augmented reality (AR) picture). For example, the picturesource 16 may be a camera used to capture a picture or a memoryconfigured to store a picture. The picture source 16 may further includeany type of (internal or external) interface that stores a previouslycaptured or generated picture and/or obtains or receives a picture. Whenthe picture source 16 is a camera, the picture source 16 may be, forexample, a local camera or an integrated camera integrated into theimage obtaining device. When the picture source 16 is a memory, thepicture source 16 may be a local memory, or may be, for example, anintegrated memory integrated into the image obtaining device. When thepicture source 16 includes an interface, the interface may be, forexample, an external interface for receiving a picture from an externalvideo source. The external video source is, for example, an externalpicture capturing device such as a camera, an external memory, or anexternal picture generation device. The external picture generationdevice is, for example, an external computer graphics processor, acomputer, or a server.

The picture may be considered as a two-dimensional array or matrix ofpixels. The pixel in the array may also be referred to as a sample. Inan optional case, to represent a color, each pixel includes three colorcomponents. For example, in an RGB format or color space, a pictureincludes corresponding red, green, and blue sample arrays. However, invideo coding, each pixel is usually represented in aluminance/chrominance format or color space, for example, a picture in aYUV format includes a luminance component indicated by Y (sometimesindicated by L alternatively) and two chrominance components indicatedby U and V. The luminance (luma) component Y represents luminance orgray level intensity (for example, both are the same in a gray-scalepicture), and the two chrominance (chroma) components U and V representchrominance or color information components. Correspondingly, thepicture in the YUV format includes a luminance sample array of luminancesample values (Y) and two chrominance sample arrays of chrominancevalues (U and V). A picture in an RGB format may be converted into apicture in a YUV format, and vice versa. This process is also referredto as color conversion or color format conversion. If a picture ismonochrome, the picture may include only a luminance sample array. Inthis embodiment of this application, a picture transmitted by thepicture source 16 to the picture processor may also be referred to asraw picture data 17.

The image processor 20 is configured to perform image processing, forexample, luminance mapping, tone mapping (tone-mapping), color formatconversion (for example, conversion from the RGB format to the YUVformat), color gamut conversion, saturation adjustment, coloradjustment, resolution adjustment, or denoising.

The communication interface 22 may be configured to receive picture data19 obtained through image processing, and may transmit, by using thelink 13 to the image processing device 14 for further image processingor to a memory for storage, the picture data 19 obtained through imageprocessing. For example, the communication interface 22 may beconfigured to encapsulate the picture data 19, obtained through imageprocessing, into a proper format such as a data packet for transmissionon the link 13.

The image processing device 14 includes a communication interface 28, animage processor 32, and the display device 34. They are separatelydescribed as follows:

The communication interface 28 may be configured to receive, from theimage obtaining device 12 or any other source, the picture data 19obtained through image processing. The any other source is, for example,a storage device. For example, for specific examples of thecommunication interface 22 and the communication interface 28, refer tothe foregoing descriptions of the interface. Details are not describedherein again. The communication interface 28 may be configured totransmit or receive, by using the link 13 between the image obtainingdevice 12 and the image processing device 14 or any other type ofnetwork, the picture data 19 obtained through image processing. Thecommunication interface 28 may be configured to, for example,decapsulate a data packet transmitted by the communication interface 22,to obtain the picture data 19 obtained through image processing.

Both the communication interface 28 and the communication interface 22may be configured as a unidirectional communication interface or abidirectional communication interface, and may be configured to, forexample, send and receive a message to establish a connection,acknowledge and exchange any other information related to acommunication link, and/or, for example, picture data obtained throughimage processing, and/or data transmission. For example, thecommunication interface 28 and the communication interface 22 may be anytype of interface based on any dedicated or standardized interfaceprotocol, for example, a high definition multimedia interface (HDMI), amobile industry processor interface (MIPI), a display serial interface(DSI) standardized by MIPI, an embedded display port (eDP), a displayport (DP), or a

V-By-One interface that is standardized by a video electronics standardsassociation (VESA), where the V-By-One interface is a digital interfacestandard developed for image transmission, and various wired or wirelessinterfaces and optical interfaces.

The image processor 32 is configured to perform tone mapping processingon the picture data 19 obtained through image processing, to obtainpicture data 33 obtained through tone mapping. Processing performed bythe image processor 32 may further include processing such as conversioninto super-resolution, color format conversion (for example, conversionfrom the YUV format to the RGB format), denoising, color gamutconversion, saturation adjustment, luminance mapping, upsampling,downsampling, and image sharpening, and the image processor 32 may befurther configured to transmit, to the display device 34, the picturedata 33 obtained through tone mapping. It should be understood that theimage processor 20 and the image processor 32 each may be a generalcentral processing unit (CPU), a system on chip (SOC), a processorintegrated into the SOC, an independent processor chip or controller, orthe like. Alternatively, the image processor 20 and the image processor32 each may be a dedicated processing device, for example, anapplication-specific integrated circuit (ASIC), a field programmablegate array (FPGA) or a digital signal processor (DSP), a dedicated videoor graphics processor, a graphics processing unit (GPU), and aneural-network processing unit (NPU). The image processor 20 and theimage processor 32 each may be a processor group including a pluralityof processors, and the plurality of processors are coupled to each otherby using one or more buses.

The display device 34 is configured to receive the picture data 33obtained through tone mapping processing, to display a picture to a useror a viewer. The display device 34 may be or include any type of displayconfigured to present a reconstructed picture, for example, anintegrated or external display or monitor. For example, the display mayinclude a liquid crystal display (LCD), an organic light emitting diode(OLED) display, a plasma display, a projector, a micro LED display, aliquid crystal on silicon (LCoS), a digital light processor (DLP), orany type of other displays. In an optional case, the display device 34has an image processing function, and the display device may performtone mapping on an image.

It should be understood that, in FIG. 1, the image obtaining device 12,the image processing device 14, and the display device 34 are drawn asseparate devices. In an optional case, an image processing apparatus mayhave functions of both the image obtaining device 12 and the imageprocessing device 14. In an optional case, an image processing apparatusmay have functions of both the image processing device 14 and thedisplay device 34. In an optional case, an image processing apparatushas functions of the image obtaining device 12, the image processingdevice 14, and the display device 34. For example, a smartphone has acamera, an image processor, and a display screen, the camera correspondsto the image obtaining device 12, corresponds to the display device 34.For another example, a smart television has a camera, an imageprocessor, and a display screen, the camera corresponds to the imageobtaining device 12, the image processor corresponds to the imageprocessing device 14, and the display screen corresponds to the displaydevice 34.

In an optional case, the image processing device includes a hardwarelayer, an operating system layer that runs above the hardware layer, andan application layer that runs above the operating system layer. Thehardware layer includes hardware such as a CPU, a memory management unit(MMU), and a memory (also referred to as a main memory). The operatingsystem may be any one or more computer operating systems that implementservice processing through a process, for example, a Linux operatingsystem, a Unix operating system, an Android operating system, an iOSoperating system, or a Windows operating system. The application layerincludes applications such as a browser, an address book, wordprocessing software, and instant messaging software. In addition, inthis embodiment of this application, no particular limitation is imposedon a specific structure of an execution body of the method provided inembodiments of this application, provided that related code can be runto implement the method provided in the embodiment of this applicationto perform image processing. For example, the method provided inembodiments of this application may be performed by an electronicdevice, or may be performed by a functional module that is in theelectronic device and that can invoke a program and execute the program,for example, may be a processor in the electronic device.

It should be understood that the image obtaining device 12 and the imageprocessing device 14 may include any type of handheld or still device,for example, a notebook or laptop computer, a mobile phone, asmartphone, a tablet or tablet computer, a video camera, a desktopcomputer, a set-top box, a television, a camera, a vehicle-mounteddevice, a display device, a digital media player, a video game console,a video streaming device (such as a content server or a contentdistribution server), a broadcast receiver device, and a broadcasttransmitter device, and may not use or may use any type of operatingsystem.

FIG. 2 is a diagram of a hardware architecture of an example imageprocessing apparatus according to an embodiment of this application. Animage processing apparatus 200 may be, for example, a processor chip.For example, the diagram of the hardware architecture shown in FIG. 2may be an example architecture diagram of the image processor 20 or theimage processor 32 in FIG. 1. The image processing method provided inembodiments of this application may be applied to the processor chip.

Referring to FIG. 2, the apparatus 200 includes: at least one CPU, amicrocontroller unit (MCU), a GPU, an NPU, a memory bus, a receiveinterface, a transmit interface, and the like. Although not shown inFIG. 2, the apparatus 200 may further include an application processor(AP), a memory, a decoder, and a dedicated video or image processor. Theforegoing parts of the apparatus 200 are coupled to each other by usinga connector. For example, the connector includes various interfaces,transmission lines, or buses. These interfaces are generally electricalcommunication interfaces, but may alternatively be mechanical interfacesor other forms of interfaces. This is not limited in this embodiment.

Optionally, the CPU may be a single-core processor (single-CPU) or amulti-core processor (multi-CPU). Optionally, the CPU may be a processorgroup including a plurality of processors, and the plurality ofprocessors are coupled to each other by using one or more buses. Thereceive interface may be a data input interface of a processor chip. Inan optional case, for types of the receive interface and the transmitinterface, refer to the descriptions in the parts of the communicationinterface 28 and the communication interface 22. Details are notdescribed herein again.

The memory is coupled to the CPU, the GPU, the MCU, the NPU, and thelike by using the memory bus. The memory may be configured to storecomputer program instructions, including a computer operating system(OS) and various user application programs. The memory is furtherconfigured to store image data or video data. Optionally, the memory maybe a nonvolatile memory, for example, an embedded multimedia card(EMMC), a universal flash storage (UFS), or a read-only memory (ROM); ormay be another type of static storage device that can store staticinformation and an instruction; or may be a volatile memory, forexample, a random access memory (RAM), or another type of dynamicstorage device that can store information and an instruction; or may bean electrically erasable programmable read-only memory (EEPROM), acompact disc read-only memory (CD-ROM) or another optical disc storage,an optical disc storage (including a compressed optical disc, a laserdisc, an optical disc, a digital versatile disc, a Blu-ray disc, or thelike), a magnetic disk storage medium or another magnetic storagedevice, or any other computer-readable storage medium that can beconfigured to carry or store program code in a form of an instruction ora data structure and can be accessed by a computer. However, the memoryis not limited thereto. It should be understood that the memory may beindependent of the image processing apparatus 200.

In an optional case, the foregoing parts are integrated into a samechip. In another optional case, the CPU, the GPU, the decoder, thereceiving interface, and the sending interface are integrated into achip, and each part in the chip accesses an external memory by using abus. The dedicated video/graphics processor may be integrated into asame chip as the CPU, or may be used as a separate processor chip. Forexample, the dedicated video/graphics processor may be a dedicated ISP.In an optional case, the NPU may also be used as a separate processorchip. The NPU is configured to implement related operations of variousneural networks or deep learning. Optionally, the image processingmethod and the image processing framework that are provided inembodiments of this application may be implemented by the GPU or theNPU, or may be implemented by a dedicated graphics processor.

A chip in embodiments of this application is a system manufactured on asame semiconductor substrate by using an integrated circuit technology,and is also referred to as a semiconductor chip. The chip may be a setof integrated circuits formed on the substrate (which is usually asemiconductor material such as silicon) by using the integrated circuittechnology, and an outer layer of the chip is usually packaged with asemiconductor packaging material. The integrated circuit may includevarious functional components. Each type of functional componentincludes a logic gate circuit, a metal oxide semiconductor (MOS)transistor, or a transistor such as a bipolar transistor or a diode, andmay also include another part such as a capacitor, a resistor, or aninductor. Each functional component may independently operate or operateunder action of necessary driver software, and may implement variousfunctions such as communication, operation, or storage.

FIG. 3 is a schematic flowchart of an image processing method accordingto an embodiment of this application. It should be understood that, forease of description, FIG. 3 describes the method by using steps. Themethod flowchart, that is, FIG. 3 shows a sequence of the method, but insome cases, the described steps can be performed in a sequence differentfrom the sequence herein.

The image processing method includes the following steps:

301. Determine a mode indication value in dynamic metadata, where themode indication value is used to indicate a target scene mode applicableto a to-be-processed image.

It should be understood that the dynamic metadata is generated when avideo front-end device generates image data or video data, and thedynamic metadata includes the mode indication value. For example, themode indication value may be obtained by the front-end device byanalyzing obtained or generated image data. The video front-end devicemay be the image obtaining device 12 shown in FIG. 1. For example, whenobtaining or generating the image data, the image obtaining device 12performs image frame analysis on each frame of image data to obtainfeature information of the frame of image data. Further, a tone mappingmode applicable to the image data may be determined based on the featureinformation obtained through analysis. For example, if an overall sceneof a current image is dark, it is determined that the image frame isapplicable to a dark-scene tone mapping mode. If an overall scene of acurrent image is bright, it is determined that the image frame isapplicable to a bright-scene tone mapping mode. If a current imageincludes a pixel located in a specific luminance range, it is determinedthat the image frame is applicable to a specific-scene tone mappingmode. Further, after receiving the to-be-processed image and the dynamicmetadata of the to-be-processed image, an image processing device 14 ora display device 34 adaptively modifies an initial tone mapping curvebased on the mode indication value in the dynamic metadata, and performstone mapping on the to-be-processed image based on the modified tonemapping curve. In an optional case, a set-top box may be referred to asa play device. It should be understood that step 301 may be performed bythe image processing device 14 or the display device 34 shown in FIG. 1.For example, the image processing device 14 may be a set-top box or animage processor in the set-top box, or may be an image processor or animage processing unit in a smart mobile terminal that has a play anddisplay capability.

302. Invoke, based on the mode indication value, an obtaining process ofa parameter that is of a modified spline curve and that corresponds tothe target scene mode, and obtain the spline curve.

Details are as follows:

When the mode indication value is a first preset value, a firstobtaining process is invoked.

When the mode indication value is a second preset value, a secondobtaining process is invoked.

When the mode indication value is a third preset value, a thirdobtaining process is invoked.

When the mode indication value is a fourth preset value, a fourthobtaining process is invoked.

When the mode indication value is a fifth preset value, a fifthobtaining process is invoked.

For example, the mode indication value included in the dynamic metadatais set to 3Spline_num.

If 3Spline_num is equal to 0, a spline curve parameter obtaining process0 is invoked, and a parameter that is of the modified spline curve andthat corresponds to a first scene mode is obtained.

If 3Spline_num is equal to 1, a spline curve parameter obtaining process1 is invoked, and a parameter that is of the modified spline curve andthat corresponds to a second scene mode is obtained.

If 3Spline_num is equal to 2, a spline curve parameter obtaining process2 is invoked, and a parameter that is of the modified spline curve andthat corresponds to a third scene mode is obtained.

If 3Spline_num is equal to 3, a spline curve parameter obtaining process3 is invoked, and a parameter that is of the modified spline curve andthat corresponds to a fourth scene mode is obtained.

If 3Spline_num is equal to 4, a spline curve parameter obtaining process4 is invoked, and a parameter that is of the modified spline curve andthat corresponds to a fifth scene mode is obtained.

In an optional case, a value of 3Spline_num may be another integergreater than or equal to 5, and corresponds to another spline curveparameter obtaining process. This is not limited in this embodiment ofthis application. Optionally, the value of 3Spline_num may be an Englishletter, a Greek letter, or another preset indication value, and is notnecessarily a Roman numeral.

Parameters of the modified spline curve that are obtained by usingdifferent spline curve parameter obtaining processes are different, andthe parameter of the modified spline curve includes a first coordinateof at least one interpolation point and a plurality of coefficients ofthe modified spline curve.

The at least one interpolation point constitutes a modified range, asub-modified range is formed between every two adjacent interpolationpoints, and the modified spline curve passes through the at least oneinterpolation point. For example, the modified spline curve may includea plurality of spline curves. When the spline curve includes a splinecurve 1 and a spline curve 2, the spline curve 1 may be an M-degreespline function in a range [x_(M1), x_(M2)], and the spline curve 2 maybe an L-degree spline function in a range [x_(L1), x_(L2)]. Anintersection set of the range [x_(M1), x_(M2)] and the range [x_(L1),x_(L2)] is empty, or the range [x_(M1), x_(M2)] and the range [x_(L1),x_(L2)] have a common endpoint value. The degree M of the spline curve 1and the degree L of the spline curve 2 may be the same or different. Forexample, the plurality of modified spline curves may include at leastone of a linear spline curve or a cubic spline curve. The modifiedspline curve may include one, two, three, or more cubic spline curves. Aquantity of cubic spline curves is not limited in this embodiment ofthis application. The cubic spline curve is smoothly connected to theinitial tone mapping curve. For example, when the modified spline curveincludes the linear spline curve and the cubic spline curve, the cubicspline curve is smoothly connected to both the linear spline curve andthe initial tone mapping curve.

A function form of the linear spline curve is y=ax+b, and the curveincludes two coefficients. At least two known conditions are requiredfor determining the linear spline curve.

A function form of a quadratic spline curve is y=ax²+bx+c, a≠0, and thecurve includes three coefficients. At least three known conditions arerequired for determining the quadratic spline curve, for example,coordinates of three points through which the quadratic spline curvepasses, or coordinates of two points that pass through the quadraticspline curve and a derivative of the quadratic spline curve at one ofthe points.

A function form of the cubic spline curve is y=ax³+bx²+cx+d, a≠0, andthe curve includes four coefficients. At least four known conditions arerequired for determining the cubic spline curve, for example,coordinates of four points through which the cubic spline curve passesthrough are obtained in advance, or coordinates of two points andderivatives of the cubic spline curve at the two points are obtained inadvance.

An N-degree spline curve includes N+1 coefficients, and at least N+1known conditions are required for determining the N-degree spline curve.

For example, FIG. 4a is a schematic diagram of an example modifiedspline curve corresponding to the foregoing first scene mode accordingto an embodiment of this application. The first scene mode correspondsto a case in which an overall scene of an image is dark.

The plurality of interpolation points include a first interpolationpoint, a second interpolation point, and a third interpolation point.The modified spline curve includes a first spline curve, a second splinecurve, and a third spline curve. The first spline curve is a linearspline curve, and both the second spline curve and the third splinecurve are cubic spline curves. The first spline curve is a straight linebetween the origin and the first interpolation point. The second splinecurve is a spline curve between the first interpolation point and thesecond interpolation point. The third spline curve is a spline curvebetween the second interpolation point and the third interpolationpoint. The first spline curve is smoothly connected to the second splinecurve at the first interpolation point. The second spline curve issmoothly connected to the third spline curve at the second interpolationpoint. The third spline curve is smoothly connected to the initial tonemapping curve at the third interpolation point. An output parameter ofthe modified spline curve includes: TH1, TH2, TH3, a slope MB0 of thefirst spline curve, four coefficients MAL MB1, MC1, and MD1 of thesecond spline curve, and four coefficients MA2, MB2, MC2, and MD2 of thethird spline curve.

The first spline curve is: F(L)=MB0×L   (1)

The second spline curve is: F(L)=MD1×(L−TH1)³ +MC1×(L−TH1)²+MB1×(L−TH1)¹ +MA1   (2)

The third spline curve is: F(L)=MD2×(L−TH2)³ +MC2×(L−TH2)² +MB2×(L−TH2)¹+MA2   (3)

TH1 is a first coordinate of the first interpolation point, TH2 is afirst coordinate of the second interpolation point, TH3 is a firstcoordinate of the third interpolation point, the first coordinate maybe, for example, a horizontal coordinate, and L is an independentvariable of a function.

The first scene mode corresponds to the case in which the overall sceneof the image is dark. In this case, a dark region is modified, to betterdisplay a dark-region detail of the image, so that a pixel whoseluminance is lower than a display capability of a display device canhave a good display effect on the display device. Specifically, tonemapping is performed, based on the linear spline function formula (1),on pixels that are lower than a first threshold (TH1), so thatdark-region detail information of the part is better reserved. Then, asmooth transition between the linear spline function and the initialtone mapping curve is implemented based on a cubic spline function, toprevent an unsmooth tone mapping curve from affecting a tone mappingeffect. It should be understood that in the first scene mode, there aretwo cubic spline curves. In an optional case, there may be only onecubic spline curve, and the first spline curve is smoothly connected tothe initial tone mapping curve by using one cubic spline function. Inthis case, only two interpolation points need to be determined. Anobtaining process in the first scene mode is the first obtainingprocess. For a detailed process of the first obtaining process, refer tothe methods shown in FIG. 6a and FIG. 6b or the following Case 3.1.Specifically, in this scenario, the first spline curve improvesdark-region luminance compared with a base curve. This is equivalent toallocating more bits to the dark region. The two cubic spline functionsimplement a smooth transition between the first spline curve and theinitial tone mapping curve, and enhancement or weakening processing maybe performed on a modified region by controlling a point on and a slopeof the curve.

FIG. 4b is a schematic diagram of an example modified spline curvecorresponding to the foregoing second scene mode according to anembodiment of this application. The second scene mode corresponds to acase in which an image includes a pixel located in a specific luminancerange.

The plurality of interpolation points include a first interpolationpoint, a second interpolation point, and a third interpolation point.The first interpolation point and the third interpolation point arelocated on the initial tone mapping curve. The second interpolationpoint may be located on the initial tone mapping curve, or may not belocated on the initial tone mapping curve. When the second interpolationpoint is not located on the initial tone mapping curve, a verticalcoordinate offset value exists between the second interpolation pointand the initial tone mapping curve. The modified spline curve includes afirst spline curve and a second spline curve. Both the first splinecurve and the second spline curve are cubic spline curves. The firstspline curve is a spline curve between the first interpolation point andthe second interpolation point. The second spline curve is a splinecurve between the second interpolation point and the third interpolationpoint. For example, the first spline curve is smoothly connected to theinitial tone mapping curve at the first interpolation point. The secondspline curve is smoothly connected to the initial tone mapping curve atthe third interpolation point. The first spline curve is smoothlyconnected to the second spline curve at the second interpolation point.An output parameter of the modified spline curve includes: TH1, TH2,TH3, four coefficients MAL MB1, MC1, and MD1 of the first spline curve,and four coefficients MA2, MB2, MC2, and MD2 of the second spline curve.

The first spline curve is: F(L)=MD1×(L−TH1)³ +MC1×(L−TH1)² +MB1×(L−TH1)¹+MA1   (4)

The second spline curve is: F(L)=MD2×(L−TH2)³ +MC2×(L−TH2)²+MB2×(L−TH2)¹ +MA2   (⁵)

TH1 is a first coordinate of the first interpolation point, TH2 is afirst coordinate of the second interpolation point, TH3 is a firstcoordinate of the third interpolation point, the first coordinate maybe, for example, a horizontal coordinate, and L is an independentvariable of a function.

The second scene mode corresponds to the case in which the imageincludes the pixel located in the specific luminance range. In thiscase, the specific luminance range is mainly modified, to better displaya luminance detail of the image in the range. Specifically, tone mappingis performed on a pixel between the first interpolation point and thesecond interpolation point and a pixel between the second interpolationpoint and the third interpolation point based on a cubic splinefunction, to improve a display effect of the part of pixels. Further, asmooth transition exists between the first spline curve and the secondspline curve and between the initial tone mapping curve and each of thefirst spline curve and the second spline curve, to prevent an unsmoothtone mapping curve from affecting a tone mapping effect. An obtainingprocess in the second scene mode is the second obtaining process. For adetailed process of the second obtaining process, refer to the methodshown in FIG. 7 or the following Case 3.3. Specifically, in thismodification manner, any range may be modified. In addition, a modifiedcurve of the modified range is smoothly connected to the initial tonemapping curve, and an interpolation point and an offset value may becontrolled to increase image contrast or increase or decrease luminancevalues of a part of pixels, to achieve a desired effect.

FIG. 4c is a schematic diagram of an example modified spline curvecorresponding to the foregoing third scene mode according to anembodiment of this application. The third scene mode corresponds to acase in which an overall scene of an image is bright.

The plurality of interpolation points include a first interpolationpoint. The modified spline curve includes a first spline curve. Thefirst spline curve is a linear spline curve that passes through thefirst interpolation point. The first spline curve is smoothly connectedto the initial tone mapping curve at the first interpolation point. Anoutput parameter of the modified spline curve includes: TH1, and a firstcoefficient MAO and a second coefficient MB0 of the first spline curve.

The first spline curve is: F(L)=MB0×(L−TH1)¹ +MA0   (6)

TH1 is a first coordinate of the first interpolation point, and thefirst coordinate may be a horizontal coordinate.

The third scene mode corresponds to the case in which the overall sceneof the image is bright. In this case, a bright region is modified, tobetter display a bright-region detail of the image. Specifically, tonemapping is performed, based on a linear function (the first splinecurve), on a pixel region that is greater than a preset threshold (TH1),and the first spline curve is smoothly connected to the initial tonemapping curve at the first interpolation point, to prevent an unsmoothtone mapping curve from affecting a tone mapping effect. Specifically,when tone mapping is performed, by using a linear spline curve, on apixel whose luminance is greater than the first coordinate of the firstinterpolation point, it indicates that pixels whose luminance is greaterthan TH1 are compressed by using a compression rate of the firstinterpolation point, to avoid excessive compression of a bright-regionpixel, and improve an image luminance value. An obtaining process in thethird scene mode is the third obtaining process. For a detailed processof the third obtaining process, refer to the method shown in FIG. 8 orthe following Case 1.

FIG. 4d is a schematic diagram of an example modified spline curvecorresponding to the foregoing fourth scene mode according to anembodiment of this application. The fourth scene mode corresponds to acase in which an overall scene of an image is bright.

The plurality of interpolation points include a first interpolationpoint and a second interpolation point. In an optional case, an offsetvalue Spline_strength exists between the second interpolation point andthe initial tone mapping curve in a vertical coordinate direction. Themodified spline curve includes a first spline curve. The first splinecurve is a cubic spline curve that passes through the firstinterpolation point and the second interpolation point. The first splinecurve is smoothly connected to the initial tone mapping curve at thefirst interpolation point. An output parameter of the modified splinecurve includes: TH1, TH2, and four coefficients MA1, MB1, MC1, and MD1of the first spline curve.

The first spline curve is: F(L)=MD1×(L−TH1)³ +MC1×(L−TH1)² +MB1×(L−TH1)¹+MA1   (⁷)

TH1 is a first coordinate of the first interpolation point, TH2 is afirst coordinate of the second interpolation point, the first coordinatemay be, for example, a horizontal coordinate, and L is an independentvariable of a function.

The fourth scene mode corresponds to the case in which the overall sceneof the image is bright. In this case, a bright region is modified, tobetter display a bright-region detail of the image. Specifically, tonemapping is performed on a pixel region between the first interpolationpoint and the second interpolation point based on a cubic splinefunction (Formula (7)), and the cubic spline function is smoothlyconnected to the initial tone mapping curve at the first interpolationpoint, to prevent an unsmooth tone mapping curve from affecting a tonemapping effect. Specifically, a luminance value of a pixel between thefirst interpolation point and a bright-region pixel endpoint may beincreased, or a luminance value of a pixel between the firstinterpolation point and a bright-region pixel endpoint may be decreased.This may achieve a more dazzling effect by increasing luminance, and mayenhance the bright-region detail by decreasing luminance. In addition,the increase and the decrease are nonlinear processes, so that contrastcan be increased. An obtaining process in the fourth scene mode is thefourth obtaining process. For a detailed process of the fourth obtainingprocess, refer to the method shown in FIG. 9 or the following Case 2.

FIG. 4e is a schematic diagram of an example modified spline curvecorresponding to the foregoing fifth scene mode according to anembodiment of this application. The fifth scene mode corresponds to acase in which an overall scene of an image is bright.

The plurality of interpolation points include a first interpolationpoint, a second interpolation point, and a third interpolation point.The modified spline curve includes a first spline curve and a secondspline curve. The first spline curve is a cubic spline curve between thefirst interpolation point and the second interpolation point. The secondspline curve is a linear spline curve between the second interpolationpoint and the third interpolation point. The first spline curve issmoothly connected to the initial tone mapping curve at the firstinterpolation point. The second spline curve is smoothly connected tothe initial tone mapping curve at the third interpolation point. Thefirst spline curve is smoothly connected to the second spline curve atthe second interpolation point. An output parameter of the modifiedspline curve includes: TH1, TH2, TH3, four coefficients MA1, MB1, MC1,and MD1 of the first spline curve, and two coefficients MA2 and MB2 ofthe second spline curve. TH1 is a first coordinate of the firstinterpolation point, TH2 is a first coordinate of the secondinterpolation point, TH3 is a first coordinate of the thirdinterpolation point, and the first coordinate may be, for example, ahorizontal coordinate.

The first spline curve is: F(L)=MD1×(L−TH1)³ +MC1×(L−TH1)² +MB1×(L−TH1)¹+MA1   (⁸)

The second spline curve is: F(L)=MB2×(L−TH2)¹ +MA2   (⁹)

L is an independent variable of a function.

It should be understood that, in the fifth scene mode, the modifiedspline curve includes one linear spline curve and one cubic splinecurve, and positions of the two spline curves are not limited. In FIG.4e , luminance of a pixel in a pixel range corresponding to the linearspline curve is greater than luminance of a pixel in a pixel rangecorresponding to the cubic spline curve, that is, a luminance value ofthe first interpolation point is less than a luminance value of thesecond interpolation point, and the luminance value of the secondinterpolation point is less than a luminance value of the thirdinterpolation point. In an optional case, luminance of a pixel in apixel range corresponding to the linear spline curve may be less thanluminance of a pixel in a pixel range corresponding to the cubic splinecurve. In this case, a luminance value of the first interpolation pointis greater than a luminance value of the second interpolation point, andthe luminance value of the second interpolation point is greater than aluminance value of the third interpolation point.

The fifth scene mode corresponds to the case in which the overall sceneof the image is bright. In this case, a bright region is modified, tobetter display a bright-region detail of the image. Specifically, tonemapping is performed on a pixel region between the second interpolationpoint and the third interpolation point based on a linear spline curve(Formula (9)), and a linear spline function is smoothly connected to theinitial tone mapping curve based on a cubic spline function between thefirst interpolation point and the second interpolation point, to preventan unsmooth tone mapping curve from affecting a tone mapping effect.Specifically, in this scenario, a cubic spline function may be usedbetween the first interpolation point and the second interpolation pointto increase luminance and decrease luminance for the initial tonemapping curve, and the increase and the decrease are nonlinear. Tonemapping is performed between the second interpolation point and thethird interpolation point based on a straight line, so that more bitscan be allocated to a pixel between the second interpolation point andthe third interpolation point, to enhance the bright-region detail. Anobtaining process in the fifth scene mode is the fifth obtainingprocess. For a detailed process of the fifth obtaining process, refer tothe method shown in FIG. 10 or the following Case 3.2.

303. Perform tone mapping on the to-be-processed image based on amodified tone mapping curve, where the modified tone mapping curveincludes at least a part of the initial tone mapping curve and themodified spline curve.

It should be understood that the initial tone mapping curve may be anyform of curve that may be used in a tone mapping process. Specifically,FIG. 4f shows several possible forms of the initial tone mapping curve.For example, the initial tone mapping curve may be a curve obtainedthrough normalization processing, that is, coordinates of any point onthe initial tone mapping curve are (first coordinate value, secondcoordinate value), a value of the first coordinate value is between 0and 1, and a value of the second coordinate value is between 0 and 1.The following describes the initial tone mapping curve obtained throughnormalization processing. It should be understood that this applicationsets no limitation thereto.

An initial tone mapping curve shown by a solid line in FIG. 4f is alsoreferred to as an SLF tone mapping curve, and meets the followingformula:

$\begin{matrix}{L^{\prime} = {{a \times \left( \frac{p \times L}{{\left( {p - 1} \right) \times L} + 1} \right)^{m}} + b}} & (10)\end{matrix}$

L represents a signal value existing before mapping, and L′ represents asignal value existing after mapping. In an optional case, the signalvalue existing before mapping is a linear signal value, the signal valueexisting after mapping is a non-linear signal value, p, a, m, and b areall dynamic range mapping coefficients, 0≤L≤1, 0≤L′≤1, a is directlyproportional to a dynamic range display capability of the displaydevice, and is inversely proportional to a dynamic range of a targetpicture, and b is directly proportional to a minimum luminance displaycapability of the display device.

An initial tone mapping curve shown by a dash-dot line in FIG. 4f isreferred to as a linear polynomial tone mapping function, and meets thefollowing formula:

L′=a×L+b   (11)

L represents a signal value existing before mapping, and L′ represents asignal value existing after mapping. In an optional case, the signalvalue existing before mapping is a linear signal value, the signal valueexisting after mapping is a non-linear signal value, both a and b aredynamic range mapping coefficients, 0≤L≤1, 0≤L′≤1, a is directlyproportional to a dynamic range display capability of the displaydevice, and is inversely proportional to a dynamic range of a targetpicture, and b is directly proportional to a minimum luminance displaycapability of the display device.

An initial tone mapping curve shown by a dashed line in FIG. 4f is alsoreferred to as a sigmoidal tone mapping curve, and meets the followingformula:

$\begin{matrix}{L^{\prime} = \frac{c_{1} + {c_{2} \times L^{n}}}{1 + {c_{3} \times L^{n}}}} & (12)\end{matrix}$

L represents a signal value existing before mapping, and L′ represents asignal value existing after mapping. In an optional case, the signalvalue existing before mapping is a linear signal value, the signal valueexisting after mapping is a non-linear signal value, c₁, c₂, c₃, and nare all dynamic range mapping coefficients, 0≤L≤1, 0≤L′≤1, and c₁, c₂,C₃, and n are all related to average luminance, maximum luminance, andminimum luminance of a target picture, and are related to minimumdisplay luminance and maximum display luminance of the display device.

An initial tone mapping curve shown by a dash-dot-dot line in FIG. 4f isalso referred to as a Bézier tone mapping curve, and meets the followingformula:

$\begin{matrix}{L^{\prime} = {\sum\limits_{\rho = 0}^{N}{\begin{pmatrix}N \\\rho\end{pmatrix} \times L^{\rho} \times \left( {1 - L} \right)^{N - \rho} \times P_{\rho}}}} & (13)\end{matrix}$

L represents a signal value existing before mapping, and L′ represents asignal value existing after mapping. In an optional case, the signalvalue existing before mapping is a linear signal value, the signal valueexisting after mapping is a non-linear signal value, n, N, and ρ are alldynamic range mapping coefficients, 0≤L≤1, 0≤L′≤1, and n, N, and ρ arerelated to luminance distribution of a target image, and are alsorelated to average luminance of the target image.

The SLF formula as an example, and p and m may be determined by usingoverall luminance of an image. When the overall image is bright, a valueof p is small, and a value of m is large. When the overall image isdark, a value of p is large, and a value of m is small. A value range ofp may be between 3 and 6. A value range of m may be between 2 and 4, andm is generally equal to 2.4.

It should be understood that the modified tone mapping curve may be apiecewise polynomial function (piecewise polynomial function). Forexample, a plurality of piecewise functions of the modified tone mappingcurve are smoothly connected to each other. Specifically, when ato-be-processed pixel is located in the modified range, tone mapping isperformed on the to-be-processed pixel based on the modified splinecurve. When a to-be-processed pixel is not located in the modifiedrange, tone mapping is performed on the to-be-processed pixel based onthe initial tone mapping curve. It should be understood that, when arange in which the to-be-processed pixel is located is determined, apixel value of the to-be-processed pixel may be compared with firstcoordinate values of a plurality of interpolation points. For example,when the to-be-processed pixel is in an RGB format, a maximum value inRGB components may be used as the pixel value of the to-be-processedpixel. When the to-be-processed pixel is in a YUV format, a Y componentmay be used as the pixel value of the to-be-processed pixel.

For example, the modified range includes a first modified range, asecond modified range, and a third modified range. The first modifiedrange corresponds to a first spline function, the second modified rangecorresponds to a second spline function, and the third modified rangecorresponds to a third spline function. In this case, when theto-be-processed pixel is located in the first modified range, tonemapping is performed on the to-be-processed pixel based on the firstspline function. When the to-be-processed pixel is located in the secondmodified range, tone mapping is performed on the to-be-processed pixelbased on the second spline function. When the to-be-processed pixel islocated in the third modified range, tone mapping is performed on theto-be-processed pixel based on the third spline function. When theto-be-processed pixel is not located in the three modified ranges, tonemapping is performed on the to-be-processed pixel based on the initialtone mapping curve.

For example, for the foregoing first scene mode, the followingprocessing is correspondingly performed:

When the to-be-processed pixel is located between the origin and thefirst interpolation point, tone mapping is performed on theto-be-processed pixel based on Formula (1).

When the to-be-processed pixel is located between the firstinterpolation point and the second interpolation point, tone mapping isperformed on the to-be-processed pixel based on Formula (2).

When the to-be-processed pixel is located between the secondinterpolation point and the third interpolation point, tone mapping isperformed on the to-be-processed pixel based on Formula (3).

When the pixel value of the to-be-processed pixel is greater than thefirst coordinate TH3 of the third interpolation point, tone mapping isperformed on the to-be-processed pixel based on the initial tone mappingcurve.

For the foregoing second scene mode, the following processing iscorrespondingly performed:

When the to-be-processed pixel is located between the firstinterpolation point and the second interpolation point, tone mapping isperformed on the to-be-processed pixel based on Formula (4).

When the to-be-processed pixel is located between the secondinterpolation point and the third interpolation point, tone mapping isperformed on the to-be-processed pixel based on Formula (5).

When the pixel value of the to-be-processed pixel is greater than thefirst coordinate TH3 of the third interpolation point, tone mapping isperformed on the to-be-processed pixel based on the initial tone mappingcurve.

For the foregoing third scene mode, the following processing iscorrespondingly performed:

When the pixel value of the to-be-processed pixel is less than the firstcoordinate TH1 of the first interpolation point, tone mapping isperformed on the to-be-processed pixel based on the initial tone mappingcurve.

When the pixel value of the to-be-processed pixel is greater than thefirst coordinate TH1 of the first interpolation point, tone mapping isperformed on the to-be-processed pixel based on Formula (6).

For the foregoing fourth scene mode, the following processing iscorrespondingly performed:

When the pixel value of the to-be-processed pixel is less than the firstcoordinate TH1 of the first interpolation point, tone mapping isperformed on the to-be-processed pixel based on the initial tone mappingcurve.

When the to-be-processed pixel is located between the firstinterpolation point and the second interpolation point, tone mapping isperformed on the to-be-processed pixel based on Formula (7).

For the foregoing fifth scene mode, the following processing iscorrespondingly performed:

When the pixel value of the to-be-processed pixel is less than the firstcoordinate TH1 of the first interpolation point, tone mapping isperformed on the to-be-processed pixel based on the initial tone mappingcurve.

When the to-be-processed pixel is located between the firstinterpolation point and the second interpolation point, tone mapping isperformed on the to-be-processed pixel based on Formula (8).

When the to-be-processed pixel is located between the secondinterpolation point and the third interpolation point, tone mapping isperformed on the to-be-processed pixel based on

Formula (9).

In the image processing method provided in this embodiment of thisapplication, a scene mode applicable to the to-be-processed image isdetermined based on the mode indication value in the dynamic metadata,and the initial tone mapping curve is properly and flexibly modified, sothat an image display effect can be adjusted for regions with differentluminance, to enhance a luminance change detail of the image, and bringbetter visual experience to a user. A plurality of styles of displayeffects such as softness, vividness, brightness, and dusk may beobtained by performing a plurality of forms of fine-tuning on the tonemapping curve.

In the image processing method provided in this embodiment of thisapplication, any part of a curve can be adaptively modified based on ascene requirement, to increase luminance, increase contrast, blurpartial information, and the like. For example, a display effect of adark-region detail is improved by increasing dark-region luminance, andcontrast may be increased by decreasing dark-region luminance, or alarge quantity of bits may be allocated to a bright region to display abright-region detail. In addition, any part of a curve may be modifiedto achieve a desired specific effect.

FIG. 5 is a flowchart of a method for obtaining a parameter of amodified spline curve according to an embodiment of this application.The method includes:

501. Determine a first coordinate of at least one interpolation pointbased on feature information or dynamic metadata of a to-be-processedimage.

For example, the at least one interpolation point may include only afirst interpolation point, or may include a first interpolation pointand a second interpolation point, or may include a first interpolationpoint, a second interpolation point, and a third interpolation point.For example, in the third scene mode in this application, only oneinterpolation point needs to be determined. In the fourth scene modeshown in this application, two interpolation points need to bedetermined. In each of the first scene mode, the second scene mode, andthe fifth scene mode in this application, three interpolation pointsneed to be determined.

502. Determine at least one second coordinate and at least onefirst-order derivative based on the first coordinate of the at least oneinterpolation point.

503. Determine coefficients of a modified spline curve based on thefirst coordinate of the at least one interpolation point, the at leastone second coordinate, and the at least one first-order derivative,where the modified spline curve includes at least one of a linear splinecurve or a cubic spline curve.

In this embodiment of this application, several example cases ofdetermining the modified spline curve are provided:

Case 1: A case in which one interpolation point is included.

The at least one interpolation point includes a first interpolationpoint. The modified spline curve includes a first spline curve. Thefirst spline curve is a linear spline curve that passes through thefirst interpolation point. The coefficients of the modified spline curveinclude a first coefficient MAO and a second coefficient MBO. The linearspline curve is smoothly connected to an initial tone mapping curve atthe first interpolation point. Specifically, a first-order derivative ofthe linear spline curve at the first interpolation point is equal to afirst-order derivative of the initial tone mapping curve at the firstinterpolation point.

Correspondingly, step 501 specifically includes: determining a firstcoordinate TH1 of the first interpolation point based on data includedin the dynamic metadata.

Step 502 specifically includes:

calculating a second coordinate VA1 of the first interpolation point onthe initial tone mapping curve and the first-order derivative GD1 of theinitial tone mapping curve at the first interpolation point based onTH1.

Step 503 specifically includes:

determining GD1 as a second coefficient MB0 of the first spline curve,and determining VA1 as a first coefficient MAO of the first splinecurve; and

obtaining the first spline curve based on MB0 and MA0.

A specific solution corresponding to Case 1 is shown in the solution inFIG. 8.

Case 2: A case in which two interpolation points are included.

The at least one interpolation point includes a first interpolationpoint and a second interpolation point. The modified spline curveincludes a first spline curve. The first spline curve is a cubic splinecurve. The second interpolation point may be one of endpoints of aninitial tone mapping curve. In the fourth scene mode shown inembodiments of this application, the second interpolation point is abright-region endpoint of the initial tone mapping curve. The modifiedspline curve is smoothly connected to the initial tone mapping curve atthe first interpolation point.

Correspondingly, step 501 specifically includes:

determining a first coordinate TH1 of the first interpolation point anda first coordinate

TH2 of the second interpolation point based on data included in thedynamic metadata. For example, TH1=A, TH2=B, where A and B are constantsincluded in the dynamic metadata, or A and B are calculated based on thedata included in the dynamic metadata.

Step 502 specifically includes:

respectively calculating second coordinates VA1 and VA2 of the firstinterpolation point and the second interpolation point on the initialtone mapping curve based on TH1 and TH2; and

calculating a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point and a first-order derivative GD2of the initial tone mapping curve at the second interpolation pointbased on TH1 and TH2.

Step 503 specifically includes:

determining VA1 as a first coefficient MA1 of the first spline curve,and determining GD1 as a second coefficient MB1 of the first splinecurve;

constructing, based on TH1, TH2, VA2, and GD2, a system of equationsthat includes two equations;

solving the system of equations to obtain a third coefficient MC1 and afourth coefficient MD1 of the first spline curve; and obtaining thefirst spline curve based on MA1, MB1, MC1, and MD1.

The constructing, based on TH1, TH2, VA2, and GD2, a system of equationsthat includes two equations specifically includes:

enabling a second coordinate of the second interpolation point on thefirst spline curve to be a sum of VA2 and S1, to obtain a firstequation, where S1 is a coordinate offset value included in the dynamicmetadata; and enabling a first-order derivative of the first splinecurve at the second interpolation point to be a sum of GD2 and S2, toobtain a second equation, where S2 is a slope offset value included inthe dynamic metadata.

A specific solution corresponding to Case 2 is shown in the solution inFIG. 9.

For the case in which two interpolation points are included, an optionalcase is further included: A first interpolation point is a point closeto a dark-region endpoint. The modified spline curve includes a firstspline curve and a second spline curve. The first spline curve is alinear spline curve. The second spline curve is a cubic spline curve.The first spline curve is a straight line that passes through the originand the first interpolation point. The second spline curve passesthrough the first interpolation point and a second interpolation point.The linear spline curve is smoothly connected to the cubic spline curveat the first interpolation point. The cubic spline curve is smoothlyconnected to an initial tone mapping curve at the second interpolationpoint. In this case, the dark region may be modified.

Case 3: A case in which three interpolation points are included.

The at least one interpolation point includes a first interpolationpoint, a second interpolation point, and a third interpolation point.

Case 3.1: The modified spline curve includes one linear spline curve andtwo cubic spline curves (the first scene mode shown in embodiments ofthis application).

Correspondingly, step 501 specifically includes:

determining a first coordinate TH1 of the first interpolation pointbased on at least one of a maximum luminance value maxL of theto-be-processed image and an average luminance value avgL of theto-be-processed image that are included in the dynamic metadata;

TH2=TH1+B; and

TH3=TH2+C*TH2−D*CH2.

TH2 is a first coordinate of the second interpolation point, TH3 is afirst coordinate of the third interpolation point, and B, C, and D areall constants. In an optional case, B, C, and D are constants includedin the dynamic metadata.

The method further includes:

obtaining a first-order derivative MB0 of a first spline curve based onthe average luminance value avgL that is of the to-be-processed imageand that is included in the dynamic metadata.

Step 502 specifically includes:

calculating a second coordinate VA1 of the first interpolation point onthe first spline curve based on TH1 and MB0;

calculating a second coordinate VA3 of the third interpolation point onan initial tone mapping curve based on TH3;

calculating a second coordinate VA2 of the second interpolation point ona first straight line, where the first straight line is a straight lineon which the first interpolation point and the third interpolation pointare located; and calculating a first-order derivative GD3 of the initialtone mapping curve at the third interpolation point.

Step 503 specifically includes:

determining VA1 as a first coefficient MA1 of a second spline curve;determining MB0 as a second coefficient MB1 of the second spline curve;

determining a sum of VA2 and an offset value as a first coefficient MA2of a third spline curve, where the offset value is 0, a positive number,or a negative number, where it should be understood that the offsetvalue may be transmitted to a tone mapping device by using the dynamicmetadata, or may be defined in program code; constructing, based on TH1,TH2, TH3, VA2, VA3, GD3, and a derivative relationship between thesecond spline curve and the third spline curve at the secondinterpolation point, a system of equations that includes five equations;and solving the system of equations that includes five equations, toobtain MC1, MD1, MB2, MC2, and MD2.

The constructing, based on TH1, TH2, TH3, VA2, VA3, GD3, and aderivative relationship between the second spline curve and the thirdspline curve at the second interpolation point, a system of equationsthat includes five equations specifically includes:

enabling a second coordinate of the second interpolation point on thesecond spline curve to be VA2, to obtain a first equation; enabling asecond coordinate of the third interpolation point on the third splinecurve to be VA3, to obtain a second equation; enabling a first-orderderivative of the third spline curve at the third interpolation point tobe GD3, to obtain a third equation;

enabling first-order derivatives of the second spline curve and thethird spline curve at the second interpolation point to be equal, toobtain a fourth equation; and

enabling second-order derivatives of the second spline curve and thethird spline curve at the second interpolation point to be equal, toobtain a fifth equation.

A specific solution corresponding to Case 3.1 is shown in the solutionsin FIG. 6a and FIG. 6 b.

Case 3.2: The modified spline curve includes a first spline curve and asecond spline curve. The first spline curve is a cubic spline curve thatpasses through the first interpolation point and the secondinterpolation point. The second spline curve is a linear spline curvethat passes through the second interpolation point and the thirdinterpolation point. The coefficients of the modified spline curveinclude a first coefficient MA1, a second coefficient MB1, a thirdcoefficient MC1, and a fourth coefficient MD1 of the first spline curve,and a first coefficient MA2 and a second coefficient MB2 of the secondspline curve (the fifth scene mode shown in embodiments of thisapplication).

Correspondingly, step 501 specifically includes:

determining a first coordinate TH1 of the first interpolation point, afirst coordinate TH2 of the second interpolation point, and a firstcoordinate TH3 of the third interpolation point based on data includedin the dynamic metadata. For example, TH1=A, TH2=B, TH3=C, where

A, B, and C are constants included in the dynamic metadata, or A, B, andC are calculated based on the data included in the dynamic metadata.

Step 502 specifically includes:

respectively calculating second coordinates VA1 and VA3 of the firstinterpolation point and the third interpolation point on an initial tonemapping curve based on TH1 and TH3, where VA1 is the first coefficientMA1 of the first spline curve;

calculating a second coordinate VA2 of the second interpolation point ona first straight line, where the first straight line is a straight lineon which the first interpolation point (TH1, VA1) and the thirdinterpolation point (TH3, VA3) are located; and

calculating a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point.

Step 503 specifically includes:

determining VA1 as the first coefficient MA1 of the first spline curve;determining GD1 as the second coefficient MB1 of the first spline curve;determining, as the first coefficient MA2 of the second spline curve, asum of VA2 and a coordinate offset value included in the dynamicmetadata, where in an optional case, the dynamic metadata includes thecoordinate offset value;

constructing, based on TH1, TH2, TH3, VA2, VA3, and a derivativerelationship between the first spline curve and the second spline curveat the second interpolation point, a system of equations that includesthree equations; and solving the system of equations that includes threeequations, to obtain MC1, MD1, and MB2.

The constructing, based on TH1, TH2, TH3, VA2, VA3, and a derivativerelationship between the first spline curve and the second spline curveat the second interpolation point, a system of equations that includesthree equations specifically includes:

enabling a second coordinate of the second interpolation point on thefirst spline curve to be VA2, to obtain a first equation;

enabling a second coordinate of the third interpolation point on thesecond spline curve to be VA3, to obtain a second equation; and

enabling first-order derivatives of the first spline curve and thesecond spline curve at the second interpolation point to be equal, toobtain a third equation.

A specific solution corresponding to Case 3.2 is shown in the solutionin FIG. 10.

Case 3.3: The modified spline curve includes a first spline curve and asecond spline curve. The first spline curve is a cubic spline curve thatpasses through the first interpolation point and the secondinterpolation point. The second spline curve is a cubic spline curvethat passes through the second interpolation point and the thirdinterpolation point. The coefficients of the modified spline curveinclude a first coefficient MA1, a second coefficient MB1, a thirdcoefficient MC1, and a fourth coefficient MD1 of the first spline curve,and a first coefficient MA2, a second coefficient MB2, a thirdcoefficient MC2, and a fourth coefficient MD2 of the second spline curve(the second scene mode shown in embodiments of this application).

Correspondingly, step 501 specifically includes:

determining a first coordinate TH1 of the first interpolation point, afirst coordinate TH2 of the second interpolation point, and a firstcoordinate TH3 of the third interpolation point based on data includedin the dynamic metadata. For example, TH1=A, TH2=B, TH3=C, where A, B,and C are constants included in the dynamic metadata, or A, B, and C arecalculated based on the data included in the dynamic metadata.

Step 502 specifically includes:

respectively calculating second coordinates VA1 and VA3 of the firstinterpolation point and the third interpolation point on an initial tonemapping curve based on TH1 and TH3;

calculating a second coordinate VA2 of the second interpolation point ona first straight line, where the first straight line is a straight lineon which the first interpolation point and the third interpolation pointare located; and

calculating a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point and a first-order derivative GD3of the initial tone mapping curve at the third interpolation point.

Step 503 specifically includes:

determining VA1 as the first coefficient MA1 of the first spline curve;

determining GD1 as the second coefficient MB1 of the first spline curve;

determining a sum of VA2 and an offset value as the first coefficientMA2 of the second spline curve, where the offset value is 0, a positivenumber, or a negative number, and optionally, the offset value may beincluded in the dynamic metadata;

constructing, based on TH1, TH2, TH3, VA2, VA3, GD3, and a derivativerelationship between the first spline curve and the second spline curveat the second interpolation point, a system of equations that includesfive equations; and

solving the system of equations that includes five equations, to obtainMC1, MD1, MB2, MC2, and MD2.

The constructing, based on TH1, TH2, TH3, VA2, VA3, GD3, and aderivative relationship between the first spline curve and the secondspline curve at the second interpolation point, a system of equationsthat includes five equations specifically includes:

enabling a second coordinate of the second interpolation point on thefirst spline curve to be VA2, to obtain a first equation;

enabling a second coordinate of the third interpolation point on thesecond spline curve to be VA3, to obtain a second equation;

enabling a first-order derivative of the second spline curve at thethird interpolation point to be GD3, to obtain a third equation;

enabling first-order derivatives of the first spline curve and thesecond spline curve at the second interpolation point to be equal, toobtain a fourth equation; and

enabling second-order derivatives of the first spline curve and thesecond spline curve at the second interpolation point to be equal, toobtain a fifth equation.

A specific solution corresponding to Case 3.3 is shown in the solutionin FIG. 7.

504. Perform tone mapping on the to-be-processed image based on amodified tone mapping curve, where the modified tone mapping curveincludes at least a part of the initial tone mapping curve and themodified spline curve.

For step 504, refer to the descriptions in the part of step 303. Detailsare not described herein again.

In an optional case, before step 501, the method further includes:

determining a target scene mode applicable to the to-be-processed imagebased on a mode indication value in the dynamic metadata, where the atleast one interpolation point and the modified spline curve are relatedto the target scene mode. Specifically, a quantity of the at least oneinterpolation point and determining of the linear spline curve and thecubic spline curve that are included in the modified spline curve arerelated to the mode indication value. Alternatively, obtaining of aparameter of a modified spline curve in a specific case in the foregoingCase 1, Case 2, Case 3.1, Case 3.2, and Case 3.3 is determined based onthe mode indication value.

FIG. 6a is a schematic flowchart of a method for obtaining a parameterof a modified spline curve according to an embodiment of thisapplication. The schematic flowchart of the method in FIG. 6a showsobtaining of a parameter of a modified spline curve in the first scenemode and Case 3.1. In this case, the modified spline curve includes onelinear spline curve, namely, a first spline curve, and two cubic splinecurves: a second spline curve and a third spline curve.

601. Obtain a first coordinate TH1 of a first interpolation point and afirst-order derivative MB0 of the first spline curve:

$\begin{matrix}{{{TH}\; 1} = \left\{ {p_{valueL} \times \left( \frac{\left( {{\max\; L} - {TPL}} \right)}{{TPH} - {TPL}} \right)\begin{matrix}p_{valueL} \\{+ p_{valueH}} \\{p_{valueH}\ }\end{matrix}\left( {1 - \left( \frac{\left( {{\max\; L} - {TPL}} \right)}{{TPH} - {TPL}} \right)} \right)\ \begin{matrix}{{\max\; L}\  > {TPH}} \\{{TPL}\  \leq {maxL}\  \leq {TPH}} \\{{maxL}\  < {TPL}}\end{matrix}} \right.} & (14) \\{{{MB}\; 0} = \left\{ {S_{valueL} \times \left( \frac{\left( {{{avg}\; L} - {AvL}} \right)}{{AvH} - {AvL}} \right)\begin{matrix}S_{valueL} \\{+ S_{valueH}} \\S_{valueH}\end{matrix} \times \left( {1 - \left( \frac{\left( {{{avg}\; L} - {AvL}} \right)}{{AvH} - {AvL}} \right)} \right)\begin{matrix}{{aveL} > {AvH}} \\{{AvL} \leq {{avg}\; L}\  \leq {AvH}} \\{{aveL} > {AvL}}\end{matrix}} \right.} & (15)\end{matrix}$

where maxL and avgL are included in dynamic metadata, maxL is a maximumluminance value of a to-be-processed image, and avgL is an averageluminance value of the to-be-processed image. When the to-be-processedimage is received, the first coordinate TH1 of the first interpolationpoint is calculated based on maxL included in the dynamic metadata ofthe to-be-processed image and Formula (14), and a slope MB0 of the firstspline curve is calculated based on avgL included in the dynamicmetadata and Formula (15). The first spline curve is, for example, thecurve shown by Formula (1). In an optional case, the first coordinate ofthe first interpolation point may be calculated based on the averageluminance value avgL of the to-be-processed image. Correspondingly, maxLin Formula (14) may be replaced with avgL. In another optional case, thefirst coordinate of the first interpolation point may be calculatedbased on the maximum luminance value maxL of the to-be-processed imageand the average luminance value avgL of the to-be-processed image.Correspondingly, maxL in Formula (14) is replaced with a*maxL+b*avgL,where a and b are both constants, 0≤a≤1, and 0≤b≤1.

Correspondingly, in an optional case, the first-order derivative MB0 ofthe first spline curve may be calculated based on the maximum luminancevalue maxL of the to-be-processed image. Correspondingly, avgL inFormula (15) may be replaced with maxL. In another optional case, thefirst coordinate of the first interpolation point may be calculatedbased on the maximum luminance value maxL of the to-be-processed imageand the average luminance value avgL of the to-be-processed image.Correspondingly, avgL in Formula (15) is replaced with a*maxL+b*avgL,where a and b are both constants, 0≤a≤1, and 0≤b≤1.

PvalueL is a first threshold of a dark-region pixel, PvalueH is a secondthreshold of the dark-region pixel, and the first threshold is less thanthe second threshold. For example, a value of PvalueL may be 0.15, and avalue of PvalueH may be 0.25. TPH is a third threshold of abright-region pixel, and TPL is a fourth threshold of the bright-regionpixel. For example, a value of TPH may be 0.6, and a value of TPL may be0.1. AvL is a fifth threshold of an average pixel, and AvH is a sixththreshold of the average pixel. For example, a value of AvL may be 0.15,and a value of AvH may be 0.65. SvalueL is a first slope threshold addedto the luminance value of the dark-region pixel, SvalueH is a secondslope threshold added to the luminance value of the dark-region pixel,and the first slope threshold is less than the second slope threshold.For example, a value of SvalueL may be 1, and a value of SvalueH may be1.2.

602. Obtain first coordinates TH2 and TH3 of a second interpolationpoint and a third interpolation point:

TH2=TH1+B   (16)

TH3=TH2+C*TH2−D*TH2   (17)

B is a first offset value of a pixel in a dark-region transition region,C is a second offset value of the bright-region pixel, D is a thirdoffset value of the bright-region pixel, and B, C, and D are constants.In an optional case, the dynamic metadata includes the foregoing B, C,and D, or B, C, and D may be directly defined in related program code,and do not need to be obtained from the dynamic metadata. For example, Bmay be 0.15, and C and D may be 0.5.

603. Determine four coefficients MA1, MB1, MC1, and MD1 of the secondspline curve and four coefficients MA2, MB2, MC2, and MD2 of the thirdspline curve based on TH1, TH2, and TH3.

It should be understood that in this case, the modified spline curveincludes two cubic spline curves: the second spline curve and the thirdspline curve. In an optional case, the modified spline curve may includeonly one cubic spline curve.

Specifically, as shown in FIG. 6b , step 603 includes:

6031. Respectively calculate second coordinates VA1, VA2, and VA3 of thefirst interpolation point, the second interpolation point, and the thirdinterpolation point.

Specifically, step 6031 includes:

(1) Calculate the second coordinate VA1 of the first interpolation pointon the linear spline curve:

VA1=MB0×TH1

(2) Calculate the second coordinate VA3 of the third interpolation pointon an initial tone mapping curve.

For example, as shown in Formula (18), it is assumed that the initialtone mapping curve is an SLF tone mapping curve:

$\begin{matrix}{{F(L)} = {{a \times \left( \frac{p \times L^{n}}{{\left( {{k_{1} \times p} - k_{2}} \right) \times L^{n}} + k_{3}} \right)^{m}} + b}} & (18)\end{matrix}$

In this case, TH3 is substituted into Formula (18) to obtain VA3=F(TH3).It should be understood that p, m, n, a, b, k₁, k₂, and k₃ in Formula(18) are all constants. A parameter of the initial tone mapping curvemay be obtained from the dynamic metadata, or may be calculated inadvance based on the to-be-processed image.

(3) Calculate the second coordinate VA2 of the second interpolationpoint on a first straight line, where the first straight line is astraight line on which the first interpolation point and the thirdinterpolation point are located:

$\begin{matrix}{{{VA}\; 2} = {{{TH}\; 2 \times \left( \frac{{{VA}\; 3} - {{TH}\; 1}}{{{TH}\; 3} - {{TH}\; 1}} \right)} + {{TH}1 \times \left( \frac{{{TH}\; 3} - {{VA}\; 3}}{{{TH}\; 3} - {{TH}\; 1}} \right)}}} & (19)\end{matrix}$

6032. Determine VA1 as the first coefficient MA1 of the second splinecurve, and determine a sum of VA2 and an offset value as the firstcoefficient MA2 of the third spline curve. Optionally, the offset valuemay be 0, a positive number, or a negative number. The offset value maybe obtained from the dynamic metadata, or may be defined in relatedprogram code of a spline curve parameter obtaining process.

6033. Determine the first-order derivative MB0 of the first spline curveas the second coefficient MB1 of the second spline curve.

6034. Calculate a first-order derivative GD3 of the initial tone mappingcurve at the third interpolation point:

$\begin{matrix}{{{GD}\; 3} = {a \times m \times p \times k_{3} \times n \times {TH}\; 3^{n - 1} \times \left( \frac{p \times {TH}\; 3^{n}}{\left. {\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 3} \right\rbrack^{n} + k_{3}} \right)^{m + 1} \times \left( \frac{1}{{TH}\; 3^{n} \times p} \right)^{2}}} & (20)\end{matrix}$

6035. Construct a system of equations based on TH1, TH2, TH3, VA2, VA3,GD3, and a derivative relationship between the second spline curve andthe third spline curve at the second interpolation point.

For example, the system of equations includes a first equation, a secondequation, a third equation, a fourth equation, and a fifth equation.Step 6035 specifically includes:

(1) Enable a second coordinate of the second interpolation point on thesecond spline curve to be VA2. In an optional case, VA2=Formula (19)+offset value, to obtain the first equation shown by Formula (21):

MD1×(TH2−TH1)³ +MC1×(TH2−TH1)²+MB1×(TH2−TH1)¹ +MA1=VA2   (21)

(2) Enable a second coordinate of the third interpolation point on thethird spline curve to be VA3, to obtain the second equation shown byFormula (22):

MD2×(TH3−TH1)³ +MC2×(TH3−TH1)² +MB2×(TH3−TH1)¹ +MA2=VA3   (22)

(3) Enable a first-order derivative of the third spline curve at thethird interpolation point to be GD3, to obtain the third equation shownby Formula (23):

3×MD2×(TH3−TH2)²+2×MC2×(TH3−TH2)¹ +MB2=GD3   (23)

(4) Enable first-order derivatives of the second spline curve and thethird spline curve at the second interpolation point to be equal, toobtain the fourth equation shown by Formula (24):

3×MD1×(TH2−TH1)²+2×MC1×(TH2−TH1)¹ +MB1=3×MD2×(TH2−TH2)²+2×MC2×(TH2−TH2)¹+MB2   (24)

(5) Enable second-order derivatives of the second spline curve and thethird spline curve at the second interpolation point to be equal, toobtain the fifth equation shown by Formula (25):

6×MD1×(TH2−TH1)¹+2×MC1=6×MD2×(TH2−TH2)¹+2×MC2   (25)

6036. Solve the system of equations that includes the first equation,the second equation, the third equation, the fourth equation, and thefifth equation, to obtain MC1, MD1, MB2, MC2, and MD2:

$\begin{matrix}{{{MC}\; 1} = {\left( {{3.0^{*}{VA}\; 2} - {2.0^{*}{GD}\; 1^{*}h\; 1} - {3.0^{*}{VA}\; 1} - {b\; 2^{*}h\; 1}} \right)/\left( {h\; 1^{*}h\; 1} \right)}} & (26) \\{{{MD}\; 1} = {\left( {{h\; 1^{*}{GD}\; 1} + {h\; 1^{*}b\; 2} + {2^{*}{VA}\; 1} - {2.0^{*}{VA}\; 2}} \right)/\left( {h\; 1^{*}h\; 1^{*}h\; 1} \right)}} & (27) \\{{{MB}\; 2} = \frac{- \begin{pmatrix}{{3.0^{*}{VA}\; 1^{*}h\; 2^{*}h\; 2} + {3.0^{*}{VA}\; 2^{*}h\; 1^{*}h\; 1} - {3.0^{*}{VA}\; 3^{*}h\; 1^{*}h\; 1} -} \\{{3.0^{*}h\; 2^{*}h\; 2^{*}{VA}\; 2} + {h\; 1^{*}h\; 2^{*}s\; 2} + {{GD}\; 1^{*}h\; 1^{*}h\; 2^{*}h\; 2}}\end{pmatrix}}{2.0^{*}h\; 2^{*}\left( {{h\; 1} + {h\; 2^{*}h\; 1}} \right)}} & (28) \\{{{MC}\; 2} = {{{MC}\; 1} + {3.0^{*}{MD}\; 1^{*}h\; 1}}} & (29) \\{{{{\left. {{{MD}\; 2} = {{{- \left( {{{VA}\; 2} - {{VA}\; 2} - {h\; 2^{*}{GD}\; 3} + {{MC}\; 1}} \right\rbrack^{*}}h\; 2^{*}h\; 2} + {3^{*}{MD}\; 1^{*}h\; 1^{*}h\; 2^{*}h\; 2}}} \right)/\left( {2^{*}h\; 2^{*}h\; 2^{*}h\; 2} \right)}\mspace{14mu}{where}\mspace{14mu} h\; 1} = {{TH}\; 2}},{{{and}\mspace{14mu} h\; 2} = {{{TH}\; 3} - {{TH}\; 2.}}}} & (30)\end{matrix}$

604. Determine the second spline curve based on MA1, MB1, MC1, and MD1,and determine the third spline curve based on MA2, MB2, MC2, and MD2.

Specifically, the second spline curve is shown by Formula (2), and thethird spline curve is shown by Formula (3).

It should be understood that, for ease of description, the method isdescribed in a form of a step in FIG. 6a and FIG. 6b . Although asequence of the method is shown in the method flowchart, in some cases,the described steps may be performed in a sequence different from thesequence herein.

FIG. 7 is a schematic flowchart of a method for obtaining a parameter ofa modified spline curve according to an embodiment of this application.The schematic flowchart of the method in FIG. 7 shows obtaining of aparameter of a modified spline curve in the second scene mode. In thiscase, the modified spline curve includes two cubic spline curves: afirst spline curve and a second spline curve. In an optional case, themodified spline curve may include only one cubic spline curve.Correspondingly, only two interpolation points need to be determined.

The method includes:

701. Obtain first coordinates TH1, TH2, and TH3 of a first interpolationpoint, a second interpolation point, and a third interpolation point.

For example, TH1=A, TH2=B, and TH3=C, where A, B, and C are allconstants. It is assumed that A and B constitute a first adjustmentrange, B and C constitute a second adjustment range, A is a minimumvalue of a luminance value of a pixel in the first adjustment range, Bis a maximum value of the luminance value of the pixel in the firstadjustment range, and C is a maximum value of a luminance value of apixel in the second adjustment range. In this case, A<B<C.

In an optional case, A, B, and C may be obtained based on dynamicmetadata. For example, the dynamic metadata includes a start location L1and an end location L3 of a range of the modified spline curve. In thiscase, the foregoing A, B, and C may be calculated based on datainformation in the dynamic metadata:

A=L1/255, C=L3/255, B=(A+C)/2

In an optional case, the dynamic metadata directly carries the foregoingA, B, and C.

In an optional case, A, B, and C may be calculated by using a formulasimilar to Formula (14).

702. Respectively calculate second coordinates VA1 and VA3 of the firstinterpolation point and the third interpolation point on an initial tonemapping curve.

For example, the initial tone mapping curve is shown by the foregoingFormula (18), TH1 is substituted into Formula (18) to obtain VA1, andTH3 is substituted into Formula (18) to obtain VA3:

$\begin{matrix}{{{VA}\; 1} = {{a \times \left( \frac{p \times {TH}\; 1^{n}}{{\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 1^{n}} + k_{3}} \right)^{m}} + b}} & (31) \\{{{VA}\; 3} = {{a \times \left( \frac{p \times {TH}\; 3^{n}}{{\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 3^{n}} + k_{3}} \right)^{m}} + b}} & (32)\end{matrix}$

703. Calculate a second coordinate VA2 of the second interpolation pointon a first straight line, where the first straight line is a straightline on which the first interpolation point and the third interpolationpoint are located.

In an optional case, a formula for calculating VA2 is the same asFormula (19).

In an optional case, the dynamic metadata includes a coordinate offsetvalue Spline_strength. It should be understood that the coordinateoffset value may be a positive number, a negative number, or 0. In thiscase, VA2 is a sum of the second coordinate of the second interpolationpoint on the first straight line and the coordinate offset value carriedin the dynamic metadata:

$\begin{matrix}{{{VA}\; 2} = {{\times \left( \frac{{{VA}\; 3} - {TH1}}{{{TH}\; 3} - {TH1}} \right)} + {{TH}\; 1 \times \left( \frac{{{TH}\; 3} - {{VA}\; 3}}{{{TH}\; 3} - {TH1}} \right)} + {Spline\_ strength}}} & (33)\end{matrix}$

704. Determine VA1 as a first coefficient MA1 of the first spline curve,and determine VA2 as a first coefficient MA2 of the second spline curve.

705. Calculate a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point and a first-order derivative GD3of the initial tone mapping curve at the third interpolation point, anddetermine GD1 as a second coefficient MB1 of the first spline curve:

$\begin{matrix}{{{GD}\; 1} = {a \times m \times p \times k_{3} \times n \times {TH}\; 1^{n - 1} \times \left( \frac{p \times {TH}\; 1^{n}}{\left. {\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 1} \right\rbrack^{n} + k_{3}} \right)^{m + 1} \times \left( \frac{1}{{TH}\; 1^{n} \times p} \right)^{2}}} & (34)\end{matrix}$

A formula for calculating GD3 is the same as Formula (20).

706. Construct a system of equations based on TH1, TH2, TH3, VA2, VA3,GD3, and a derivative relationship between the first spline curve andthe second spline curve at the second interpolation point, where thesystem of equations includes five equations.

For example, step 706 specifically includes:

(1) Enable a second coordinate of the second interpolation point on thefirst spline curve to be VA2, to obtain a first equation, where thefirst equation is shown by Formula (21).

(2) Enable a second coordinate of the third interpolation point on thesecond spline curve to be VA3, to obtain a second equation shown byFormula (22).

(3) Enable a first-order derivative of the second spline curve at thethird interpolation point to be GD3, to obtain a third equation shown byFormula (23).

(4) Enable first-order derivatives of the first spline curve and thesecond spline curve at the second interpolation point to be equal, toobtain a fourth equation shown by Formula (24).

(5) Enable second-order derivatives of the first spline curve and thesecond spline curve at the second interpolation point to be equal, toobtain a fifth equation shown by Formula (25).

707. Solve the system of equations that includes the first equation, thesecond equation, the third equation, the fourth equation, and the fifthequation, to obtain MC1, MD1, MB2, MC2, and MD2.

For a calculation result, refer to Formula (26) to Formula (30). Detailsare not described herein again.

708. Determine the first spline curve based on MA1, MB1, MC1, and MD1,and determine the second spline curve based on MA2, MB2, MC2, and MD2.

The first spline curve is: F(L)=MD1×(L−TH1)³ +MC1×(L−TH1)² +MB1×(L−TH1)¹+MA1   (35)

The second spline curve is: F(L)=MD2×(L−TH2)³ +MC2×(L−TH2)²+MB2×(L−TH2)¹ +MA2   (36)

In this scenario, the first coordinates of the first interpolationpoint, the second interpolation point, and the third interpolation pointare obtained based on the dynamic metadata, and both the firstinterpolation point and the third interpolation point are located on theinitial tone mapping curve. The dynamic metadata may further include avertical coordinate offset value of the second interpolation point, andthe vertical coordinate offset value is used to indicate an offsetvalue, of the second interpolation point, that is relative to a straightline defined by the first interpolation point and the thirdinterpolation point and that is in a vertical coordinate direction.

FIG. 8 is a schematic flowchart of a method for obtaining a parameter ofa modified spline curve according to an embodiment of this application.The schematic flowchart of the method in FIG. 8 shows obtaining of aparameter of a modified spline curve in the third scene mode. In thiscase, the modified spline curve includes one linear spline curve: afirst spline curve.

A parameter that needs to be determined includes a first coordinate TH1of a first interpolation point, and a first coefficient MAO and a secondcoefficient MB0 of the first spline curve.

The method includes:

801. Obtain the first coordinate TH1 of the first interpolation point.

In an optional solution, TH1=A.

Optionally, A may be a constant carried in dynamic metadata, or A may becalculated based on data carried in dynamic metadata. For example, ifthe dynamic metadata carries a start value L1 of a modified range,A=L1/255.

In an optional solution, TH1 may be calculated based on featureinformation of a picture and a piecewise formula similar to Formula(14).

802. Calculate a second coordinate VA1 of the first interpolation pointon an initial tone mapping curve.

The initial tone mapping curve is shown by Formula (18), and TH1 issubstituted into Formula (18) to obtain VA1:

$\begin{matrix}{{{VA}\; 1} = {{a \times \left( \frac{p \times {TH}\; 1^{n}}{{\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 1^{n}} + k_{3}} \right)^{m}} + b}} & (37)\end{matrix}$

803. Calculate a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point:

$\begin{matrix}{{{GD}\; 1} = {a \times m \times p \times k_{3} \times n \times {TH}\; 1^{n - 1} \times \left( \frac{p \times {TH}\; 1^{n}}{\left. {\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 1} \right\rbrack^{n} + k_{3}} \right)^{m + 1} \times \left( \frac{1}{{TH}\; 1^{n} \times p} \right)^{2}}} & (38)\end{matrix}$

804. Determine GD1 as the second coefficient MB0 of the first splinecurve, and determine VA1 as the first coefficient MAO of the firstspline curve.

Correspondingly, the obtained modified spline curve is shown by Formula(6).

FIG. 9 is a schematic flowchart of a method for obtaining a parameter ofa modified spline curve according to an embodiment of this application.The schematic flowchart of the method in FIG. 9 shows obtaining of aparameter of a modified spline curve in the fourth scene mode. In thiscase, the modified spline curve includes one cubic spline curve: a firstspline curve.

The method includes:

901. Obtain first coordinates TH1 and TH2 of a first interpolation pointand a second interpolation point based on dynamic metadata.

For example, TH1=A, and TH2=B, where A and B are constants. For example,B may be 1.0.

In an optional case, A and B may be directly obtained from the dynamicmetadata. In an optional case, A and B may be calculated based on datainformation carried in the dynamic metadata. Alternatively, A and B maybe calculated based on a feature of a to-be-processed image and apiecewise formula similar to Formula (14).

902. Respectively calculate second coordinates VA1 and VA2 of the firstinterpolation point and the second interpolation point on an initialtone mapping curve.

For example, the initial tone mapping curve is shown by the foregoingFormula (18), TH1 is substituted into Formula (18) to obtain VA1, andTH2 is substituted into Formula (18) to obtain VA2:

$\begin{matrix}{{{VA}\; 1} = {{a \times \left( \frac{p \times {TH}\; 1^{n}}{{\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 1^{n}} + k_{3}} \right)^{m}} + b}} & (39) \\{{{VA}\; 2} = {{a \times \left( \frac{p \times {TH}\; 2^{n}}{{\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 2^{n}} + k_{3}} \right)^{m}} + b}} & (40)\end{matrix}$

903. Calculate a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point and a first-order derivative GD2of the initial tone mapping curve at the second interpolation point:

$\begin{matrix}{{{GD}\; 1} = {a \times m \times p \times k_{3} \times n \times {TH}\; 1^{n - 1} \times \left( \frac{p \times {TH}\; 1^{n}}{\left. {\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 1} \right\rbrack^{n} + k_{3}} \right)^{m + 1} \times \left( \frac{1}{{TH}\; 1^{n} \times p} \right)^{2}}} & (41) \\{{{GD}\; 2} = {a \times m \times p \times k_{3} \times n \times {TH}\; 2^{n - 1} \times \left( \frac{p \times {TH}\; 2^{n}}{\left. {\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 2} \right\rbrack^{n} + k_{3}} \right)^{m + 1} \times \left( \frac{1}{{TH}\; 2^{n} \times p} \right)^{2}}} & (42)\end{matrix}$

904. Determine VA1 as a first coefficient MA1 of the first spline curve,and determine GD1 as a second coefficient MB1 of the first spline curve.

905. onstruct a system of equations based on TH1, TH2, VA2, and GD2.

For example, step 905 specifically includes:

(1) Enable a second coordinate of the second interpolation point on thefirst spline curve to be VA2′=VA2+S1, to obtain a first equation shownby Formula (43):

MD1×(TH2−TH1)³ +MC1×(TH2−TH1)² +MB1×(TH2−TH1)¹ +MA1=VA2+S1   (43)

S1 is a coordinate offset value included in the dynamic metadata.Optionally, S1 may be a positive value, a negative value, or 0.

(2) Enable a first-order derivative of the first spline curve at thesecond interpolation point to be GD2′=GD2+S2, to obtain a secondequation shown by Formula (44):

3×MD1×(TH2−TH1)²+2×MC1×(TH2−TH1)¹ +MB1=GD2+S2   (44)

S2 is a slope offset value included in the dynamic metadata. Optionally,S2 may be a positive value, a negative value, or 0.

906. Solve the system of equations that includes the first equation andthe second equation, to obtain MC1 and MD1.

Correspondingly, the obtained modified spline curve is shown by Formula(45):

F(L)=MD1×(L−TH1)³ +MC1×(L−TH1)² +MB1×(L−TH1)¹ +MA1   (45)

FIG. 10 is a schematic flowchart of a method for obtaining a parameterof a modified spline curve according to an embodiment of thisapplication. The schematic flowchart of the method in FIG. 10 showsobtaining of a parameter of a modified spline curve in the fifth scenemode. In this case, the modified spline curve includes one cubic splinecurve and one linear spline curve. A parameter of the modified splinecurve includes: four coefficients MA1, MB1, MC1, and MD1 of a firstspline curve, and two coefficients MA2 and MB2 of a second spline curve.

The method includes:

1001. Obtain first coordinates TH1, TH2, and TH3 of a firstinterpolation point, a second interpolation point, and a thirdinterpolation point.

For example, TH1=A, TH2=B, and TH3=C, where A, B, and C are allconstants. It is assumed that A and B constitute a first adjustmentrange, B and C constitute a second adjustment range, A is a minimumvalue of a luminance value of a pixel in the first adjustment range, Bis a maximum value of the luminance value of the pixel in the firstadjustment range or a minimum value of a luminance value of a pixel inthe second adjustment range, and C is a maximum value of the luminancevalue of the pixel in the second adjustment range. In this case, A<B<C.

In an optional case, A, B, and C may be obtained based on dynamicmetadata. For example, the dynamic metadata includes a start location L1and an end location L3 of a range of the modified spline curve. In thiscase, the foregoing A, B, and C may be calculated based on datainformation in the dynamic metadata:

A=L1/255, C=L3/255, B=(A+C)/2

In an optional case, the dynamic metadata directly carries the foregoingA, B, and C.

In an optional case, A, B, and C may be calculated by using a formulasimilar to Formula (14).

1002. Respectively calculate second coordinates VA1 and VA3 of the firstinterpolation point and the third interpolation point on an initial tonemapping curve.

For details, refer to the descriptions in the part of step 702. Detailsare not described herein again.

1003. Calculate a second coordinate VA2 of the second interpolationpoint on a first straight line, where the first straight line is astraight line on which the first interpolation point (TH1, VA1) and thethird interpolation point (TH3, VA3) are located.

In an optional case, a formula for calculating VA2 is the same asFormula (19).

In an optional case, the dynamic metadata includes a coordinate offsetvalue Spline_strength. It should be understood that the coordinateoffset value may be a positive number, a negative number, or 0. WhenSpline_strength is 0, a value of VA2 may be calculated based on Formula(19). In this case, VA2 is a sum of the second coordinate of the secondinterpolation point on the first straight line and the coordinate offsetvalue carried in the dynamic metadata:

$\begin{matrix}{{{VA}\; 2} = {{\times \left( \frac{{{VA}\; 3} - {{TH}\; 1}}{{{TH}\; 3} - {{TH}\; 1}} \right)} + {{TH}1 \times \left( \frac{{{TH}\; 3} - {{VA}\; 3}}{{{TH}\; 3} - {{TH}\; 1}} \right)} + {Spline\_ strength}}} & (46)\end{matrix}$

It should be understood that VA1<VA2<VA3.

1004. Determine VA1 as the first coefficient MA1 of the first splinecurve, and determine VA2 as the first coefficient MA2 of the secondspline curve.

1005. Calculate a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point and a first-order derivative GD3of the initial tone mapping curve at the third interpolation point, anddetermine GD1 as the second coefficient MB1 of the first spline curve:

$\begin{matrix}{{{GD}\; 1} = {a \times m \times p \times k_{3} \times n \times {TH}\; 1^{n - 1} \times \left( \frac{p \times {TH}\; 1^{n}}{\left. {\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 1} \right\rbrack^{n} + k_{3}} \right)^{m + 1} \times \left( \frac{1}{{TH}\; 1^{n} \times p} \right)^{2}}} & (47) \\{{{GD}\; 3} = {a \times m \times p \times k_{3} \times n \times {TH}\; 3^{n - 1} \times \left( \frac{p \times {TH}\; 3^{n}}{\left. {\left( {{k_{1} \times p} - k_{2}} \right) \times {TH}\; 3} \right\rbrack^{n} + k_{3}} \right)^{m + 1} \times \left( \frac{1}{{TH}\; 3^{n} \times p} \right)^{2}}} & \left( {48} \right)\end{matrix}$

1006. Construct a system of equations based on TH1, TH2, TH3, VA2, VA3,and a derivative relationship between the first spline curve and thesecond spline curve at the second interpolation point.

For example, step 1006 specifically includes:

(1) Enable a second coordinate of the second interpolation point on thefirst spline curve to be VA2, to obtain a first equation shown byFormula (49):

MD1×(TH2−TH1)³ +MC1×(TH2−TH1)² +MB1×(TH2−TH1)¹ +MA1=VA2   (49)

(2) Enable a second coordinate of the third interpolation point on thesecond spline curve to be VA3, to obtain a second equation shown byFormula (50):

MB2×(TH3−TH2)¹ +MA2=VA3   (50)

(3) Enable first-order derivatives of the first spline curve and thesecond spline curve at the second interpolation point to be equal, toobtain a third equation shown by Formula (51):

3×MD1×(TH2−TH1)²+2×MC1×(TH2−TH1)¹ +MB1=MB2   (51)

1007. Solve the system of equations that includes the first equation,the second equation, and the third equation, to obtain MC1, MD1, andMB2.

The obtained modified spline curve is shown by Formula (52) and Formula(53), where Formula (52) is the first spline curve, and Formula (53) isthe second spline curve:

F(L)=MD1×(L−TH1)³ +MC1×(L−TH1)² +MB1×(L−TH1)¹ +MA1   (52)

F(L)=MB2×(L−TH2)¹ +MA2   (53)

It should be understood that, for ease of description, the method isdescribed in a form of a step in the method flowcharts shown in FIG. 7to FIG. 10. Although a sequence of the method is shown, in some cases,the described steps may be performed in a sequence different from thesequence herein.

FIG. 11 is a structural block diagram of an example image processingapparatus 1100 according to an embodiment of this application. The imageprocessing apparatus 1100 includes:

a determining module 1110, configured to perform steps 301, 302, 501,502, and 503 in the method embodiments; and a tone mapping module 1120,configured to perform steps 303 and 504 in the method embodiments.

Specifically, the determining module is further configured to performsome or all of the steps in the method embodiments shown in FIG. 6 a,FIG. 6 b, and FIG. 7 to FIG. 10.

In an optional case, the image processing apparatus 1100 furtherincludes an obtaining module 1130, configured to obtain ato-be-processed image and corresponding dynamic metadata.

It should be understood that the image processing device 14 shown inFIG. 1 and the image processing apparatus shown in FIG. 2 may beconfigured to implement any one of the method embodiments provided inFIG. 3, FIG. 5, FIG. 6 a, FIG. 6 b, and FIG. 7 to FIG. 10 in embodimentsof this application. Specifically, the image processing device 14 andthe processor in FIG. 2 invoke program instructions stored in a memory,to perform some or all of the steps in the method in any one ofembodiments in FIG. 3, FIG. 5, FIG. 6 a, FIG. 6 b, and FIG. 7 to FIG.10.

An embodiment of this application further provides a computer-readablestorage medium. The computer-readable storage medium storesinstructions. When the computer-readable storage medium is run on acomputer or a processor, the computer or the processor is enabled toperform one or more steps in any one of the foregoing methods. When themodules in the foregoing image processing apparatus are implemented inthe form of a software functional unit and sold or used as anindependent product, the unit may be stored in the computer-readablestorage medium.

Based on such an understanding, an embodiment of this applicationfurther provides a computer program product including instructions. Whenthe computer program product runs on a computer or a processor, thecomputer or the processor is enabled to perform any method provided inembodiments of this application. The technical solutions of thisapplication essentially, or the part contributing to the conventionaltechnology, or all or some of the technical solutions may be implementedin a form of a software product. The computer software product is storedin a storage medium, and includes several instructions for instructing acomputer device or a processor in a computer device to perform all orsome of the steps of the methods in embodiments of this application.

A person of ordinary skill in the art may be aware that units andalgorithm steps in the examples described with reference to embodimentsdisclosed in this specification may be implemented by electronichardware or a combination of computer software and electronic hardware.Whether the functions are performed by hardware or software depends onparticular applications and design constraints of the technicalsolutions. A person skilled in the art may use different methods toimplement the described functions of each particular application, but itshould not be considered that the implementation goes beyond the scopeof this application.

The foregoing embodiments are merely intended for describing thetechnical solutions of this application, but not for limiting thisapplication. Although this application is described in detail withreference to the foregoing embodiments, persons of ordinary skill in theart should understand that they may still make modifications to thetechnical solutions described in the foregoing embodiments or makeequivalent replacements to some technical features thereof, withoutdeparting from the scope of the technical solutions of embodiments ofthis application.

What is claimed is:
 1. An image processing method, comprising:determining a first coordinate of at least one interpolation point basedon feature information or dynamic metadata of a to-be-processed image;determining at least one second coordinate and at least one first-orderderivative corresponding to the first coordinate of the at least oneinterpolation point; determining coefficients of a modified spline curvebased on the first coordinate of the at least one interpolation point,the at least one second coordinate, and the at least one first-orderderivative, wherein the modified spline curve comprises at least one ofa linear spline curve or a cubic spline curve; and performing tonemapping tone-mapping on the to-be-processed image based on a modifiedtone mapping curve, wherein the modified tone mapping curve comprises atleast a part of an initial tone mapping curve and the modified splinecurve.
 2. The method according to claim 1, wherein the at least oneinterpolation point comprises a first interpolation point and a secondinterpolation point, the modified spline curve comprises a first splinecurve, the first spline curve is a cubic spline curve that passesthrough the first interpolation point and the second interpolationpoint, and the coefficients of the modified spline curve comprise afirst coefficient MA1, a second coefficient MB1, a third coefficientMC1, and a fourth coefficient MD1; the determining at least one secondcoordinate and at least one first-order derivative based on the firstcoordinate of the at least one interpolation point comprises:respectively calculating second coordinates VA1 and VA2 of the firstinterpolation point and the second interpolation point on the initialtone mapping curve; and calculating a first-order derivative GD1 of theinitial tone mapping curve at the first interpolation point and afirst-order derivative GD2 of the initial tone mapping curve at thesecond interpolation point. the determining coefficients of a modifiedspline curve based on the first coordinate of the at least oneinterpolation point, the at least one second coordinate, and the atleast one first-order derivative comprises: determining VA1 as a firstcoefficient MA1 of the first spline curve; determining GD1 as a secondcoefficient MB1 of the first spline curve; and constructing, based onTH1, TH2, VA2, and GD2, a system of equations; and solving the system ofequations to obtain a third coefficient MC1 and a fourth coefficient MD1of the first spline curve.
 3. The method according to claim 1, whereinthe at least one interpolation point comprises a first interpolationpoint, the modified spline curve comprises a first spline curve, thefirst spline curve is a linear spline function that passes through anorigin and the first interpolation point, and the coefficients of themodified spline curve comprise MB0 of the first spline curve; before thedetermining at least one second coordinate and at least one first-orderderivative based on the first coordinate of the at least oneinterpolation point, the method further comprises: obtaining MB0 of thefirst spline curve based on at least one of a maximum value maxL ofluminance of the to-be-processed image and an average value avgL ofluminance of the to-be-processed image that are comprised in the dynamicmetadata.
 4. The method according to claim 3, wherein the at least oneinterpolation point further comprises a second interpolation point and athird interpolation point, the modified spline curve further comprises asecond spline curve and a third spline curve, the second spline curve isa cubic spline curve that passes through the first interpolation pointand the second interpolation point, the third spline curve is a cubicspline curve that passes through the second interpolation point and thethird interpolation point, and the coefficients of the modified splinecurve further comprise: MA1, MB1, MC1, MD1, MA2, MB2, MC2 and MD2; thedetermining at least one second coordinate and at least one first-orderderivative based on the first coordinate of the at least oneinterpolation point comprises: calculating a second coordinate VA1 ofthe first interpolation point on the first spline curve based on a firstcoordinate TH1 of the first interpolation point and MB0; calculating asecond coordinate VA3 of the third interpolation point on the initialtone mapping curve based on a first coordinate TH3 of the thirdinterpolation point; calculating a second coordinate VA2 of the secondinterpolation point on a first straight line based on a first coordinateTH2 of the second interpolation point, wherein the first straight lineis a straight line on which the first interpolation point and the thirdinterpolation point are located; and calculating a first-orderderivative GD3 of the initial tone mapping curve at the thirdinterpolation point based on TH3.
 5. The method according to claim 4,wherein the determining coefficients of a modified spline curve based onthe first coordinate of the at least one interpolation point, the atleast one second coordinate, and the at least one first-order derivativecomprises: determining VA1 as MA1; determining MB0 as MB1; determining asum of VA2 and an offset value as MA2, wherein the offset value is 0, apositive number, or a negative number; constructing, based on TH1, TH2,TH3, VA2, VA3, GD3, and a derivative relationship between the secondspline curve and the third spline curve at the second interpolationpoint, a system of equations; and solving the system of equations, toobtain MC1, MD1, MB2, MC2, and MD2.
 6. The method according to claim 5,wherein the system of equations comprises:MD1×(TH2−TH1)³ +MC1×(TH2−TH1)² +MB1×(TH2−TH1)¹ +MA1=VA2MD2×(TH3−TH1)³ +MC2×(TH3−TH1)² +MB2×(TH3−TH1)¹ +MA2=VA33×MD2×(TH3−TH2)²+2×MC2×(TH3−TH2)¹ +MB2=GD33×MD1×(TH2−TH1)²+2×MC1×(TH2−TH1)¹ +MB1=3×MD2×(TH2−TH2)^(2+2×)MC2×(TH2−TH2)¹ +MB26×MD1×(TH2−TH1)¹+2×MC1=6×MD2×(TH2−TH2)¹+2×MC2 wherein,3×MD1×(TH2−TH1)²+2×MC1×(TH2−TH1)¹+MB1 represents the first-orderderivative of the second spline curve at the second interpolation point,3×MD2×(TH2−TH2)²+2×MC2×(TH2−TH2)¹+MB2 represents the first-orderderivative of the third spline curve at the second interpolation point;6×MD1×(TH2−TH1)¹+2×MC1 represents the second-order derivative of thesecond spline curve at the second interpolation point,6×MD2×(TH2−TH2)¹+2×MC2 represents the second-order derivative of thethird spline curve at the second interpolation point.
 7. The methodaccording to claim 4, wherein the determining a first coordinate of atleast one interpolation point based on feature information or dynamicmetadata of a to-be-processed image comprises: determining the firstcoordinate TH1 of the first interpolation point based on at least one ofa maximum value maxL of luminance of the to-be-processed image and anaverage value avgL of luminance of the to-be-processed image that arecomprised in the dynamic metadata; TH2=TH1+B; and TH3=TH2+C*TH2−D*CH2,wherein B, C, and D are constants.
 8. The method according to claim 1,wherein the at least one interpolation point comprises a firstinterpolation point, a second interpolation point, and a thirdinterpolation point, the modified spline curve comprises a first splinecurve and a second spline curve, the first spline curve is a cubicspline curve that passes through the first interpolation point and thesecond interpolation point, the second spline curve is a cubic splinecurve that passes through the second interpolation point and the thirdinterpolation point, and the coefficients of the modified spline curvecomprise: MA1, MB1, MC1, MD1, MA2, MB2, MC2 and MD2; the determining atleast one second coordinate and at least one first-order derivativebased on the first coordinate of the at least one interpolation pointcomprises: respectively calculating second coordinates VA1 and VA3 ofthe first interpolation point and the third interpolation point on theinitial tone mapping curve based on a first coordinate TH1 of the firstinterpolation point and a first coordinate TH3 of the thirdinterpolation point; calculating a second coordinate VA2 of the secondinterpolation point on a first straight line based on a first coordinateTH2 of the second interpolation point, wherein the first straight lineis a straight line on which the first interpolation point and the thirdinterpolation point are located; and calculating a first-orderderivative GD1 of the initial tone mapping curve at the firstinterpolation point and a first-order derivative GD3 of the initial tonemapping curve at the third interpolation point.
 9. The method accordingto claim 8, wherein the determining coefficients of a modified splinecurve based on the first coordinate of the at least one interpolationpoint, the at least one second coordinate, and the at least onefirst-order derivative comprises: determining VA1 as MA1; determiningGD1 as MB1; and determining a sum of VA2 and an offset value as MA2,wherein the offset value is 0, a positive number, or a negative number;constructing, based on TH1, TH2, TH3, VA2, VA3, GD3, and a derivativerelationship between the first spline curve and the second spline curveat the second interpolation point, a system of equations; and solvingthe system of equations, to obtain MC1, MD1, MB2, MC2, and MD2.
 10. Themethod according to claim 1, wherein before the determining a firstcoordinate of at least one interpolation point based on featureinformation or dynamic metadata of a to-be-processed image, the methodfurther comprises: determining a target scene mode applicable to theto-be-processed image based on a mode indication value in the dynamicmetadata, wherein the at least one interpolation point and the modifiedspline curve are related to the target scene mode.
 11. An imageprocessing apparatus, comprising a processor and a transmissioninterface, wherein the transmission interface is configured to receiveor send image data; and the processor is configured to: invoke programinstructions stored in a memory, and execute the program instructions toperform: determining a first coordinate of at least one interpolationpoint based on feature information or dynamic metadata of ato-be-processed image; determining at least one second coordinate and atleast one first-order derivative corresponding to the first coordinateof the at least one interpolation point; determining coefficients of amodified spline curve, wherein the modified spline curve comprises atleast one of a linear spline curve or a cubic spline curve; andperforming tone mapping tone-mapping on the to-be-processed image basedon a modified tone mapping curve, wherein the modified tone mappingcurve comprises at least a part of an initial tone mapping curve and themodified spline curve.
 12. The image processing apparatus according toclaim 11, wherein the at least one interpolation point comprises a firstinterpolation point and a second interpolation point, the modifiedspline curve comprises a first spline curve, the first spline curve is acubic spline curve that passes through the first interpolation point andthe second interpolation point, and the coefficients of the modifiedspline curve comprise a first coefficient MA1, a second coefficient MB1,a third coefficient MC1, and a fourth coefficient MD1; wherein theprocessor is further configured to perform: respectively calculatingsecond coordinates VA1 and VA2 of the first interpolation point and thesecond interpolation point on the initial tone mapping curve; andcalculating a first-order derivative GD1 of the initial tone mappingcurve at the first interpolation point and a first-order derivative GD2of the initial tone mapping curve at the second interpolation point. thedetermining coefficients of a modified spline curve based on the firstcoordinate of the at least one interpolation point, the at least onesecond coordinate, and the at least one first-order derivativecomprises: determining VA1 as a first coefficient MA1 of the firstspline curve; determining GD1 as a second coefficient MB1 of the firstspline curve; and constructing, based on TH1, TH2, VA2, and GD2, asystem of equations; and solving the system of equations to obtain athird coefficient MC1 and a fourth coefficient MD1 of the first splinecurve.
 13. The image processing apparatus according to claim 11, whereinthe at least one interpolation point comprises a first interpolationpoint, the modified spline curve comprises a first spline curve, thefirst spline curve is a linear spline function that passes through anorigin and the first interpolation point, and the coefficients of themodified spline curve comprise MB0; wherein the processor is furtherconfigured to perform: obtaining MB0 based on at least one of a maximumvalue maxL of luminance of the to-be-processed image and an averagevalue avgL of luminance of the to-be-processed image that are comprisedin the dynamic metadata.
 14. The image processing apparatus according toclaim 13, wherein the at least one interpolation point further comprisesa second interpolation point and a third interpolation point, themodified spline curve further comprises a second spline curve and athird spline curve, the second spline curve is a cubic spline curve thatpasses through the first interpolation point and the secondinterpolation point, the third spline curve is a cubic spline curve thatpasses through the second interpolation point and the thirdinterpolation point, and the coefficients of the modified spline curvefurther comprise: MA1, MB1, MC1, MD1, MA2, MB2, MC2 and MD2; wherein theprocessor is further configured to perform: calculating a secondcoordinate VA1 of the first interpolation point on the first splinecurve based on a first coordinate TH1 of the first interpolation pointand MB0; calculating a second coordinate VA3 of the third interpolationpoint on the initial tone mapping curve based on a first coordinate TH3of the third interpolation point; calculating a second coordinate VA2 ofthe second interpolation point on a first straight line based on a firstcoordinate TH2 of the second interpolation point, wherein the firststraight line is a straight line on which the first interpolation pointand the third interpolation point are located; and calculating afirst-order derivative GD3 of the initial tone mapping curve at thethird interpolation point based on TH3.
 15. The image processingapparatus according to claim 14, wherein the processor is furtherconfigured to perform: determining VA1 as MA1; determining MB0 as MB1;determining a sum of VA2 and an offset value as MA2, wherein the offsetvalue is 0, a positive number, or a negative number; constructing, basedon TH1, TH2, TH3, VA2, VA3, GD3, and a derivative relationship betweenthe second spline curve and the third spline curve at the secondinterpolation point, a system of equations; and solving the system ofequations, to obtain MC1, MD1, MB2, MC2, and MD2.
 16. The imageprocessing apparatus according to claim 12, wherein the processor isfurther configured to perform: determine a first coordinate TH1 of thefirst interpolation point based on at least one of a maximum value maxLof luminance of the to-be-processed image and an average value avgL ofluminance of the to-be-processed image that are comprised in the dynamicmetadata; TH2=TH1+B; and TH3=TH2+C*TH2−D*CH2, wherein B, C, and D areconstants.
 17. A computer-readable storage medium, wherein thecomputer-readable storage medium stores instructions; and when theinstructions are run on a computer or a processor, the computer or theprocessor is enabled to perform: determining a first coordinate of atleast one interpolation point based on feature information or dynamicmetadata of a to-be-processed image; determining at least one secondcoordinate and at least one first-order derivative corresponding to thefirst coordinate of the at least one interpolation point; determiningcoefficients of a modified spline curve, wherein the modified splinecurve comprises at least one of a linear spline curve or a cubic splinecurve; and performing tone mapping tone-mapping on the to-be-processedimage based on a modified tone mapping curve, wherein the modified tonemapping curve comprises at least a part of an initial tone mapping curveand the modified spline curve.
 18. The computer-readable storage mediumaccording to claim 17, wherein the at least one interpolation pointcomprises a first interpolation point, the modified spline curvecomprises a first spline curve, the first spline curve is a linearspline function that passes through an origin and the firstinterpolation point, and the coefficients of the modified spline curvecomprise MB0; wherein the computer or the processor is further enabledto perform: obtaining MB0 based on at least one of a maximum value maxLof luminance of the to-be-processed image and an average value avgL ofluminance of the to-be-processed image that are comprised in the dynamicmetadata.
 19. The computer-readable storage medium according to claim18, wherein the at least one interpolation point further comprises asecond interpolation point and a third interpolation point, the modifiedspline curve further comprises a second spline curve and a third splinecurve, the second spline curve is a cubic spline curve that passesthrough the first interpolation point and the second interpolationpoint, the third spline curve is a cubic spline curve that passesthrough the second interpolation point and the third interpolationpoint, and the coefficients of the modified spline curve furthercomprise: MA1, MB1, MC1, MD1, MA2, MB2, MC2 and MD2; wherein thecomputer or the processor is further enabled to perform: calculating asecond coordinate VA1 of the first interpolation point on the firstspline curve based on a first coordinate TH1 of the first interpolationpoint and MB0; calculating a second coordinate VA3 of the thirdinterpolation point on the initial tone mapping curve based on a firstcoordinate TH3 of the third interpolation point; calculating a secondcoordinate VA2 of the second interpolation point on a first straightline based on a first coordinate TH2 of the second interpolation point,wherein the first straight line is a straight line on which the firstinterpolation point and the third interpolation point are located; andcalculating a first-order derivative GD3 of the initial tone mappingcurve at the third interpolation point based on TH3.
 20. Thecomputer-readable storage medium according to claim 19, wherein thecomputer or the processor is further enabled to perform: determining VA1as MA1; determining MB0 as MB1; determining a sum of VA2 and an offsetvalue as MA2, wherein the offset value is 0, a positive number, or anegative number; constructing, based on TH1, TH2, TH3, VA2, VA3, GD3,and a derivative relationship between the second spline curve and thethird spline curve at the second interpolation point, a system ofequations; and solving the system of equations, to obtain MC1, MD1, MB2,MC2, and MD2.